Archives for the day of: April 18, 2012

Over the years, many have dreamed about using mobile technology to create a kind of “augmented reality translation system” that translates whatever you are looking at into a language you can understand, and displays it in place.  Back in 2008, Intel’s Paul Otellini demonstrated a prototype Chinese-to-English translator as part of his CES Keynote.   More recently, Word Lens for the iPhone does a beautiful job of cleanly translating text in place (supporting english, spanish and french), but the translation (in my experience) tends to be a bit spotty.

Now, Microsoft has demonstrated a far more comprehensive approach as part of Bing Translate for Windows Mobile, and it looks amazing.

[msn c0b91059-9ce0-477f-a391-53fbb53c061e 534 300]

I’ll be trying it out on one of our Nokia 800′s, and it along would probably make it worth carrying the phone on international trips, even though I’m still firmly an iPhone user.  If they keep putting amazing things like this out, and decide to add in better non-MS-centric mail and calendar to WP8, and perhaps that will change.

Ok, I know I’m either going to love or hate a blog post about an internal research project on mobile AR, when the first line is

It all started with Snow Crash.

Michael Abrash, game and tech industry veteran, and now at Valve, recently wrote a blog post about his journey to Valve and the current internal research project on what he calls “wearable computing.”   For the rest of us, though, the description of what they are doing sounds exactly what researchers like me would call fully immersive AR. (Ironically, the folks at Google are doing something that the research community would traditionally call wearable computing, and folks want to call it AR, but that’s a different conversation.)

Suffice to say, I loved this post.  Now, he doesn’t really provide much information on what they are doing, so it’s not that it’s the details that are what’s compelling.  But, I find myself really excited about the idea of a top tier game company doing an internal project that is focused on what is arguably the distant future of AR gaming.

Sure, companies like Sony and Nintendo and Microsoft have built AR games for recently, and the core feature of the PS Vita, 3DS and Kinect is the ability to do augmented reality style games.  But, all of these products and projects are focused on the near future, on games and experiences that will be possible with this hardware. In my research group, especially our Augmented Reality Game Studio, we’ve been working on similar kinds of games for many years, and so I obviously see the value in trying to figure out how to create compelling AR game experiences with current hardware and software (heck, we will hopefully be releasing one of our games, Nerdherder, into the app stores soon!)

But, lately I’ve been thinking that it’s time to refocus on the future.  When we started the AR Game Studio, it still wasn’t easy to build AR games on mobile devices, and we were lucky to team up with Qualcomm to see what would be possible with what has since become Vuforia, their free AR SDK (that is probably one of the best solutions for doing AR on mobiles right now).  Now that Vuforia is out there, and is available for the Unity game engine as a free plugin, everyone can get in on the action!  If you can build games in Unity, you can build AR games!    I will keep teaching AR game design classes, and working with students in my lab to build these games.  And my students and I will keep investigating how to create compelling experiences of this sort (with the advantage that we can study what others make, too, not just what we’ve built).

But, my first love is working with technology and concepts on the cutting edge that are (for all intents and purposes) impossible on a large scale, rather than studying and understanding what people can do right now.

Which brings me back to Michael Abrash.  I have no idea what they are really doing.  But, the thing I’ve learned about game design over the past half dozen years, building games in my lab and at my company, is that game design is different than research, even if your research needs you to build a game.  Unlike researchers or product engineers, game designers are trained and focused on creating compelling experiences.  Things only matter insofar as they are perceived by the player and impact the experience of the player.  If the experience sucks, the experience sucks:  there is no “oh, it only works if you do this” or “imagine what it would be like if this was better or that was better”.  There is no making it work well enough for the paper, or the video, or the evaluation.  There is only the experience.   As Miyagi said in The Karate Kid:

Miyagi: Now, ready?
Daniel: Yeah, I guess so.
Miyagi: Daniel-san, must talk. Walk on road, hm? Walk left side, safe. Walk right side, safe. Walk middle, sooner or later, (makes squish gesture) get squish just like grape. Here, karate, same thing. Either you karate do “yes”, or karate do “no”. You karate do “guess so”, (makes squish gesture) just like grape. Understand?

AR (or wearable computing, as Michael calls it) will not really be possible any time soon.  To safely and unobtrusively integrate content with your view of the world around you, you must be able to achieve the kind of integration we see in live sports on TV, and that simply isn’t going to be feasible soon.  But if you want to work towards that dream, you need to decide:  am I building something to take me one step in that direction (e.g., my games in the AR Game Studio, or release of the Argon AR-enabled web browser, Google’s Project Glass) or are we going to just try to create a compelling experience of what it might be like when we get there.

I don’t know if the later is what Valve really wants to do, but I think a game studio that is saying up front

To be clear, this is R&D – it doesn’t in any way involve a product at this point, and won’t for a long while, if ever – so please, no rumors about Steam glasses being announced at E3.

is likely to do something interesting and exciting and different than the other projects you see around the web.

Too bad Valve isn’t in Atlanta.

In light of all the talk of Google’s project glass, there are a bunch of folks suggesting that the other displays on the market can already do what Project Glass is trying to do.  And, I will agree, some of these displays are getting closer.

Brother’s Airscouter (apparently due out this summer in Japan) is indeed a see-through virtual retinal display, and is “relatively small”.

YouTube Preview Image

But, it’s a LOT bigger and bulkier, and it’s “just the display.”   No camera.  No computer.  No sensors.  So, in practice, it would be much bigger, and a lot uglier.  As a researcher, of course, I’m totally fine with this, since it will let us do the research we want with the platforms we want (from desktop to laptop to tablet to handheld) and not be bound to a specific OS or set of sensors.  But, for practical use, that’s a huge problem.  So, we’ll likely be buying some (there’s nothing else quite like it coming out), just as we’ll be buying other displays!

Epson has an actual see-through display on the market, and it’s relatively cheap!

Like Project Glass, the Moverio BT-100 includes an Android device and is see-though.  But, it’s also stereo!   Unfortunately, it doesn’t include sensors on the display (either a camera or orientation sensors to tell which direction the head is looking), so it can’t really be used for augmented reality.  I am not even sure if it includes GPS (to let it at least be used for a limited collection of location aware applications, like Project Glass does).  So, again,  this isn’t really useful for the kinds of things we want to do in AR or even wearable computing.  The integration and reliance on an Android device (running a relatively old version of Android) is also a huge problem for folks like me, who want to try new things.  We are porting our AR-enabled web browser (Argon) to Android right now, but will only support ICS (Android 4.0) and above; in fact, since there are bugs in the Android webview that won’t be fixed till at least the next version (I hope) we probably can’t even support 4.0.  Having to use 2.2 or other old revs of Android isn’t that useful.

So, the world keeps moving forward, things keep getting better … but we aren’t quite there yet!