Blaise Aguera y Arcas demos augmented-reality maps

About a year and a half ago, Stephen Lawler, who also gave a talk here at TED in 2007 on Virtual Earth, brought me over to become the architect of Bing Maps, which is Microsoft's online-mapping effort. In the past two and a half, we've been very hard at work on redefining the way maps work online. And we really are seeing this in very different terms from the kind of mapping and direction site that one is used to.

So, the first thing that you might notice about the mapping site is just the fluidity of the zooming and the panning, which, if you're familiar at all with Seadragon, that's where it comes from.

Mapping is, of course, not just about cartography, it's also about imagery. So, as we zoom-in beyond a certain level this resolves into a kind of Sim City-like virtual view at 45 degrees. This can be viewed from any of the cardinal directions to show you the 3D structure of the city, all the facades. Now, we see this space, this three-dimensional environment, as being a canvas on which all sorts of applications can play out, and map's directions are really just one of them.

If you click on this, you'll see some of the ones that we've put out, just in the past couple of months since we've launched. So, for example, a couple of days after the disaster in Haiti, we had an earthquake map that showed before and after pictures from the sky. This wonderful one which I don't have time to show you is taking hyper-local blogs in real time and mapping those stories, those entries to the places that are referred to on the blogs. It's wonderful.

But I'm going to show you some more candy sort of stuff.

So, we see the imagery, of course, not stopping at the sky. These little green bubbles represent photosynths that users have made. I'm not going to dive into them either, but photosynths are integrated into the map. Everything that's cased in blue is an area where we've taken imagery on the ground as well.

And so, when you fly down --

(Applause)

Thank you. When you fly down to the ground, and you see this kind of panoramic imagery, the first thing that you might notice is that it's not just a picture, there's just as much three-dimensional understanding of this environment as there is of the three-dimensional city from above, so if I click on something to get a closer view of it, then, the fact that that transition looks as it does, is a function of all of that geometry, all of that 3D understanding behind this model.

Now, I'll show you a fun app that -- we've been working on a collaboration with our friends at Flickr. This takes Flickr, georegistered imagery and uses photosynth-like processes to connect that imagery to our imagery, so -- I'm not sure if that's the one I actually meant to pull up, but --

(Laughter)

But notice -- this is, of course, a popular tourist site, and there are lots of photos around here, and these photos are all taken at different times. So this one was taken around five. So that's the Flickr photo, that's our imagery. So you really see how this kind of crowd-sourced imagery is integrating, in a very deep way, into the map itself.

(Applause)

Thank you.

(Applause)

There are several reasons why this is interesting and one of them, of course, is time travel. And I'm not going to show you some of the wonderful historic imagery in here, but there are some with horses and carriages and so on as well. But what's cool about this is that, not only is it augmenting this visual representation of the world with things that are coming in from users, but it also is the foundation for augmented reality, and that's something that I'll be showing you more of in just a moment.

Now I just made a transition indoors. That's also interesting. OK, notice there's now a roof above us. We're inside the Pike Place Market. And this is something that we're able to do with a backpack camera, so, we're now not only imaging in the street with this camera on tops of cars, but we're also imaging inside. And from here, we're able to do the same sorts of registration, not only of still images, but also of video.

So this is something that we're now going to try for the first time, live, and this is really, truly, very frightening.

(Laughter)

OK.

(Ringing)

All right, guys, are you there?

(Noise)

All right. I'm hitting it. I'm punching play. I'm live. All right. There we go. So, these are our friends in Pike Place Market, the lab.

(Applause)

So they're broadcasting this live. OK, George, can you pan back over to the corner market? Because I want to show points of interest. No, no. The other way. Yeah, yeah, back to the corner, back to the corner. I don't want to see you guys yet. OK, OK, back to the corner, back to the corner, back to the corner. OK, never mind.

What I wanted to show you was these points of interest over here on top of the image because what that gives you a sense of is the way, if you're actually on the spot, you can think about this -- this is taking a step in addition to augmented reality.

What the hell are you guys -- oh, sorry.

(Laughter)

We're doing two different -- OK, I'm hanging up now. We're doing two different things here. One of them is to take that real ...

(Laughter)

All right, let me just take a moment and thank the team. They've done a fantastic job of pulling this together.

(Applause)

I'm going to abandon them now and walk back outside. And while I walk outside, I'll just mention that here we're using this for telepresence, but you can equally well use this on the spot, for augmented reality. When you use it on the spot, it means that you're able to bring all of that metadata and information about the world to you. So here, we're taking the extra step of also broadcasting it. That was being broadcast, by the way, on a 4G network from the market.

All right, and now there's one last TED talk that Microsoft has given in the past several years. And that's Curtis Wong, WorldWide Telescope. So, we're going to head over to the dumpsters, where it's traditional, after a long day at the market, to go out for a break, but also stare up at the sky. This is the integration of WorldWide Telescope into our maps.

(Applause)

This is the current -- thank you -- this is the current time. If we scrub the time, then we can see how the sky will look at different times, and we can get all of this very detailed information about different times, different dates: Let's move the moon a little higher in the sky, maybe change the date. I would like to kind of zoom in on the moon. So, this is an astronomically complete representation of the sky integrated right into the Earth.

All right now, I've overrun my time, so I've got to stop. Thank you all very much.

(Applause)