Johnny Lee demos Wii Remote hacks

So, as researchers, something that we often do is use immense resources to achieve certain capabilities, or achieve certain goals. And this is essential to the progress of science, or exploration of what is possible. But it sort of creates this unfortunate situation where a tiny, tiny fraction of the world can actually participate in this exploration or can benefit from that technology. And something that motivates me, and what gets me really excited about my research, is when I see simple opportunities to drastically change that distribution and make the technology accessible to a much wider percentage of the population.

And I'm going to show you two videos that have gotten a lot of attention recently that I think embody this philosophy. And they actually use the Nintendo Wii remote. Now, for those of you who aren't familiar with this device, it's a $40 video game controller. And it's mostly advertised for its motion sensor capabilities: so you can swing a tennis racket, or hit a baseball bat. But what actually interests me a lot more is the fact that in the tip of each controller is a relatively high-performing infrared camera. And I'm going to show you two demos of why this is useful.

So here, I have my computer set up with the projector, and I have a Wii remote sitting on top of it. And, for example, if you're in a school that doesn't have a lot money, which is probably a lot of schools, or if you're in an office environment, and you want an interactive whiteboard, normally these cost about two to three thousand dollars. So what I'm going to show you how to do is how to create one with a Wii remote. Now, this requires another piece of hardware, which is this infrared pen. You can probably make this yourself for about five dollars with a quick trip to the Radio Shack. It's essentially got a battery, a button and an infrared LED, and it turns on -- you guys can't see it -- but it turns on whenever I push the button. Now, what this means is that if I run this piece of software, the camera sees the infrared dot, and I can register the location of the camera pixels to the projector pixels. And now this is like a whiteboard surface. (Applause) So for about $50 of hardware, you can have your own whiteboard. This is Adobe Photoshop. (Applause) Thank you. (Laughter)

Now, the software for this I've actually put on my website and have let people download it for free. And in the three months that this project has been public, it's been downloaded over half a million times. So teachers and students all around the world are already using this. (Applause) I want to quickly say that although it does do it for 50 dollars, there are some limitations of this approach. But you get about 80 percent of the way there, for about one percent of the cost. Another nice thing is that a camera can see multiple dots, so this is actually a multi-touch, interactive whiteboard system as well. (Applause)

For the second demo, I have this Wii remote that's actually next to the TV. So it's pointing away from the display, rather than pointing at the display. And why this is interesting is that if you put on, say, a pair of safety glasses, that have two infrared dots in them, what these two dots are essentially going to give you is, give the computer an approximation of your head location. And why this is interesting is I have this sort of application running on the computer monitor, which has a 3D room, with some targets floating in it. And you can see that it looks like a 3D room -- if you can see -- kind of like a video game, it sort of looks 3D, but for the most part, the image looks pretty flat, and bound to the surface of the screen. But if we turn on head tracking, the computer can change the image that's on the screen and make it respond to the head movements. So let's switch back to that. (Laughter) (Applause)

So this has actually been a little bit startling to the game development community. (Laughter) Because this is about 10 dollars of additional hardware if you already have a Nintendo Wii. So I'm looking forward to seeing some games, and actually Louis Castle, that's him down there, last week announced that Electronic Arts, one of the largest game publishers, is releasing a game in May that has a little Easter egg feature for supporting this type of head tracking. So -- and that's from less than five months from a prototype in my lab to a major commercial product. (Applause) Thank you.

But actually, to me, what's almost more interesting than either of these two products is how people actually found out about them. YouTube has really changed the way, or changed the speed, in which a single individual can actually spread an idea around the world. You know, I'm doing some research in my lab with a video camera, and within the first week, a million people had seen this work, and literally within days, engineers, teachers and students from around the world, were already posting their own YouTube videos of them using my system or derivatives of this work. So I hope to see more of that in the future, and hope online video distribution to be embraced by the research community. So thank you very much. (Applause)