BrianPeek.com

WARNING! This thread is not really about Brian Peek's WiimoteLib.

Hey guys,
I'm posting this here because I haven't managed to find any general "usage of wiimote on the PC" forums anywhere. And I know that this forum is monitored by people like me, who are interested in this topic - and thanks to Brian Peek for the API that helps us achieve this :)

Does anyone know if there have ever been made a free program for the PC which can be used to create and recognize gestures using the wiimote (on a PC)? 
What I am thinking about is pretty much the same functionality as the following commercial product existing for the Wii:
http://www.ailive.net/liveMovePro.html
If it does not exist (as I don't think it does) it is my plan to make it... Would anyone maybe be interested in such a tool?

The reason why I want to develop it is that I need this kind of functionality for a game I am going to create in about half a year, and then I figured that I might as well let other people gain access to the tool if they can use it for anything.Both the tool to create/edit gestures and the “gesture recognition component” (to be used during runtime) is being made in C# (yes I have begun the work). I am (of course) using the WiimoteLib to get the sensor data from the Wiimote.

try wiigee in sourceforge.net, it is a free wiimote gesture library for java...i'm also interested in gesture recognition using wiimote and i hv try to built a c# program myself. but i found some difficulties in it..so when u'll finish your tool??

Thanks for the hint, I will check out wiigee and see if it is up to the job :)
But I will still make the tool myself because I will much rather have it in C# than Java... I'm handing in my master thesis this friday so I guess I would not be able to work so much on the tool until then, but I hope I can get a stable beta version ready sometime in late september.
Can I ask what problems you faced when trying to develop such a tool, was it with the pattern recognition algorithm? And did you use a special kind of algorithm to recognize the gestures?

yes the problem is in gesture recognition..i dont know much about it, it's something about probabilistic, data clustering,etc..and yes i also think its better i c# since right until now i'm using c# in my project.

oh, and how far has u made in your project for this tool? are u using brian's wiimote library too?

Yes I'm using Brian's library to get the Wiiremote input data, it works great... The Wiiremote/bluetooth/Windows issue is more annoying, I mean the fact that you have to "Search for new Bluetooth device" each time you connect the Wiiremote to your PC is pretty irritating (IMO). It would be great if you could just press buttons 1+2 while your program was running and then it would be found and be ready for use right away :)

I have made allmost everything NOT related to the exact gesture recognition... I do know that the missing part is exactly the core of the tool and the hardest part to make, but I thought it would be easier to test the gesture recognition if the rest of the program was running first. And by the rest of the program I mean that there is some functionality to record gestures and see the data on a chart, edit/delete data afterwards.
I have experience with quite a lot of AI algorithms (and to some extent some pattern recognition) so I hope I will manage to figure out some kind of clever recognition algorithm :)

 

 

with AI algorithms i think u use neural network, i guess..but in other project i see a lot of people is using ' hidden markov model 'instead of neural network. i dont know which is better, i  think u should know which..keep posting the progress ok, and i thought that tool would be very usefull for other people..

Hey guys.

I created the beginnings of a gesture recognition system in C#/WPF and began some very basic toolsets like using some graphs so you can view x,y,z positioning of the gesture etc.

I based my implementation on some whitepapers & examples I found talking about OCR. Once I started reading about OCR it's really not a whole lot different than motion gestures except its a 3 axis gesture instead of 2. Although I suck at math, I was able to make it "work" but its flawed as I think my math during implementation went wrong somewhere for sure. I added some hacks to stabalize it a bit, but I think its still not so accurate.

The stuff you see from that company though is not just gesture recognition, with the MotionPlus they now achieve 1:1 motion mapping. I am REALLY interested in this as well.

Gesture recognition does not give you a ton of flexibility depending on what your trying to achieve.

I think about WiiTennis. How amazing would it be if you actually had full control of your swing. Being able to slice, drop shots, actually aim the ball. That is where I began. Trying to see what gestures I could recognize but soon learned if I wanted to understand full motions, you would have to implement almost a flight simulator against the accelerometer.

Does anyone know what the MotionPlus actually does? I'm hoping it simplifies the developers lives. If its just more axis then we still need a flight simulator like backend to really understand the orientation of the device.

I think I would be willing to join in and possibly contribute all of my work. I haven't touched it in awhile and perhaps having more of a group to work with would be appealing to further it.

Also a very interesting thing. You mentioned wiigee. After looking at wiigee it looks like it uses a library called bluecove to interface with bluetooth. bluecove supports a bunch of OS's and BlueTooth stacks.

From briefly reading I am guessing wiigee does all the BT pairing itself. I haven't tried it yet since its late and I just saw it from your post but if its supporting all these stacks, and we can programatically in java do the pairing I think this is very valuable to the WiiMote since this is the biggest hurdle for casual PC users to getting the WiiMote to work on their PC.

So 2 things come to mind:
1. We could tie the two together. Some sort of java WiiMote server or some such which does the pairing/feeding info to C#.
2. Even better, a project called IKVM. It's basically a JVM implemented in .NET which means java code will run ontop of .NET without needing java installed. I wonder if the apis bluecove uses is implemented in IKVM and we can just piggy back the bluecove library right in C# to initiate the discovery/pairing.

It's also possible that I am wrong that it does the pairing. I just figured this because of the stacks they say it supports. 

Any thoughts on this Brian?

Synced: I also heard from a guy who did a Java game supporting Wiimotes that it was possible to run his game, and then press 1+2 and then it automatically installed/paired the Wiimote directly ingame, without any need to manually search for bluetooth devices.... I didn't really believe him, but it might actually be the case that it works with some of the Java API's. I agree that it would be a GREAT advantage if people would not need to get into the "advanced" stuff of having to connect the Wiimote manually to the PC. It would make the games a lot more accessible.

 What kind of algorithm did you use to recognize gestures?

 AFAIK the motionplus is just sending updates to the host a lot more frequently, meaning that you don't "loose" any measurements due to too fast movements. But this will not remove the issue that you don't really know where the Wiimote is located relative to the user. Except if you ask the user to hold the Wiimote at a certain position and press a specific button in the beginning of the game, and you never loose track of any movements (and he actually holds the mote at the right position when syncing).

 

It's believable because most Bluetooth stacks have an SDK so that you can code against it. Most people are discouraged about having to support all the stacks if they want to implement this kind of thing because people could buy any device. It would be worth looking into though if this library does do all the pairing.

I very quickly looked at the java lib and it's basically hooking directly into WIDCOMM, BlueSoleil, etc. to do the pairing.  This can also be done on the .NET side as I've said, but it requires someone to write the code to do so against each stack separately.  There's no common interface to do this sort of thing.

I plan on looking into the MS and WIDCOMM stacks at some point to see what exactly is required, but I don't have any dongles that use any other stacks, so it'd be up to the community to write rest.

Brian: If you do the first of them (or both MS and Widcomm) I really think it would be possible to get some of "the rest of us" to help with the others. The persons having other stacks should be very interested in getting it to work with their stack, at least I would. 

The instant I finish the book, I'll be circling back to look at this stuff again.  Probably in a couple weeks...

http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html

I tested it here and it work pretty well.

This thing does not use the Wiimote but rather a normal VGA camera.

Regards.