Archives for category: demos

Over the years, many have dreamed about using mobile technology to create a kind of “augmented reality translation system” that translates whatever you are looking at into a language you can understand, and displays it in place.  Back in 2008, Intel’s Paul Otellini demonstrated a prototype Chinese-to-English translator as part of his CES Keynote.   More recently, Word Lens for the iPhone does a beautiful job of cleanly translating text in place (supporting english, spanish and french), but the translation (in my experience) tends to be a bit spotty.

Now, Microsoft has demonstrated a far more comprehensive approach as part of Bing Translate for Windows Mobile, and it looks amazing.

[msn c0b91059-9ce0-477f-a391-53fbb53c061e 534 300]

I’ll be trying it out on one of our Nokia 800′s, and it along would probably make it worth carrying the phone on international trips, even though I’m still firmly an iPhone user.  If they keep putting amazing things like this out, and decide to add in better non-MS-centric mail and calendar to WP8, and perhaps that will change.

Ori posted a video of the demos, over on his blog.  He called it the first AR demo at GDC, which I don’t think is really true.  Sony must have demonstrated Eye of Judgement years ago, for example.  Plus, NVidia is demoing our game on their Tegra in their booth (so, we at least tied!).  :)

But, I don’t think any of that is important.  The Metaio demos are nice, and it’s good to see them combined with the Vuzix displays in the GDC expo!

Here’s a direct link to the video Ori took:

YouTube Preview Image

There’s been enough press about this, so I won’t go into details.  Vuzix had a demo of their two newest displays (the 920, with the camera attachment, and a not-yet-functional physical prototype of the Wrap, due out this summer).  Suffice to say, I believe these displays are marking a turning point for mobile AR. 

Are they sleek and stylish?  Hmmm.  Not yet (by fashion standards) … but, by “what came before them” standards, the new wrap will be very interesting.  Will I carry a pair and wear them?  The answer there depends 100% on the answer to the question “What will I see when I look through them?”   In other words, what are the applications going to be?

The important point, of course, is that the price and quality of these displays, combined with the power of the next generation of mobile devices, will make it possible for people to finally start answering that question.

Blair in HMD.

Ok, I’ve tried to avoid focusing on my own work here, but it’s been a fun week at GDC 2009 so far (the Game Developer’s Conference).  I gave a talk on Handheld AR Games on Monday at the GDC Mobile Summit (Ori wrote about that), and met a lot of interesting people after it.  There is clearly a growing understanding that AR is going to be a significant feature in the landscape of mobile AR, which is really exciting!

I also met with some folks from TouchArcade, a really great site for iPhone game reviews.  We talked for a while about our little iPhone game demo,and I showed them the other demo game we created on NVidia’s reference platform. They wrote a small story about our chat; the big point is that, while the current AR games on mobiles are still a bit rough around the edges, when you see what the new platforms (whether NVidia’s Tegra or TI’s OMAP3 or Qualcomm’s SnapDragon), you begin to understand what handheld AR might be like!

Not to leave out some video, here’s a short video of what I showed them on the NVidia Tegra.

YouTube Preview Image

And, for reference, our older iPhone pet game:

YouTube Preview Image

It really is such a shame to see such a potentially good idea miss the mark.  A bunch of stories are running around the blog-o-sphere today, likely fueled by the article in the New York Times about Topps’ idea to save the baseball trading card business: AR baseball cards (go here and here and here, for example.  I like the blogs, since many of the comments express my feelings with much more colorful language).

 The idea is seemingly simple, to associate some virtual content with the card so that it serves as a bridge to the virtual world. It looks like an application tailor made for Total Immersion’s software (or software from a number of other folks, like Metaio).  Of course, a few clever Georgia Tech (or Graz and Oxford or Cambridge or MIT or …) students could hack the underlying system together in a few weeks with the Flash AR Toolkit, a bit of OpenCV and a database (of course, building the content is another story!). 

The problem with this concept is not the technology (although the graphics are pretty lame).  Rather, it is the failure to seemingly even consider trying to answer the question I drill into my students:  So What?  Why would anyone care? Putting a pointless, cute little 3D model on an advertisement is actually not a bad idea, since the technology is young and the point of advertising is to generate a bit of buzz and hopefully some associated mindshare.  

But, trading cards are different.  Here, the card is the business, and you are in effect asking people to spend real money for what is nothing more than a cute gimmick.   To quote the NYT article:

Total Immersion, a French company, brought Topps the augmented reality technology. It has already been used in a theme park and for some auto design work. Using the technology, card collectors see a three-dimensional version of a player and can play elementary pitching, batting and catching games using the computer keyboard.

Seeing 3D versions of your favorite players is … ummm … cute.  But there are more fun ways to do that (can you say “XBox 360″ or “PS3″?), and these have fun (not “elementary”) games associated with them. 

The NYT article quotes Eisner (of post-Disney fame) as being behind the idea;  does this mean he actually thinks people (kids, collectors?) will pay money for this?  I suspect, instead, this is a classic example of folks sitting around saying “they will buy this” even though the people coming up with the idea wouldn’t themselves buy it.  

Mr. Eisner said Topps expected to ship 10 million packs of Series 1 (12 cards for $2) and Topps Attax cards this year (5 for $1). Scott Kelnhofer, editor of Card Trade, an industry publication, says the Total Immersion technology could strike a chord with boys. “This is the boldest technology idea we’ve seen in sports cards so far. The key is not to have it be a novelty and then it’s on to the next one.”

Bold does not equal good.  As I say to the students in my class when they do experience or game designs:  if you are designing for some imagined person who “likes doing that stuff” (whatever that stuff is) even though you think it’s not fun, you are not going to succeed.  Or, to use a simpler HCI concept “get to know your user, don’t just trust your instincts”.  Stereotypes are fun, but they shouldn’t be the basis of a business plan.

There is, of course, room for this to grow into something interesting.  But, not by following this route.  I look forward to the day when AR is commonplace and a wide range of creative professionals are empowered to develop ideas.  Right now, we’re stuck in the AR-on-a-marker mindset (yes, these cards are effectively just pretty versions of the much-maligned ugly-black-and-white marker), where the “reality” is left out of “augmented reality”.  Perhaps we should start calling this “augmented video”?  

Of course, you might say “talk is cheap”.  Do I have big ideas on how to improve on this concept?  Sure, but I’ll reserve my detailed critique on how to improve this idea for in-class discussion (we’re teaching an AR Design class right now, no point giving away what our excellent Georgia Tech students are paying for!), in-lab brainstorming (my research group needs to publish, after all) and my consulting work (well, of course!).   But the basics are obvious:  leverage the real world, provide something of value that the users could not get any other way.  Here, the card could just link to a web site with the 3D model, and with video and stats and fun (not “elementary”) games.  Putting the little model on a card adds nothing, and (much like Eye of Judgement) will not hold collectors attention.

Of course, perhaps I’ll be wrong.  Perhaps this should be viewed as an elaborate advertising campaign?  If they can get a small percentage of current and potential customers to buy a pack or two, the whole thing may pay for itself and help them limp through these troubled times.  On the surface, however, taking the idea as it’s being presented, it’s a dud.

This videos are pretty slick.  While it might be possible to nitpic on some of them, as a whole they are very cool and compelling.  There seems to be the Surface group’s “fingerprints” all over them (lots of them appear to be Surface in different form factors), and some of it is a bit implausible, but over all it’s cool to really see what a lot of our ideas might look like.  After all, form factor is important, and its often hard to imagine what something would be like in practice when the current implementation is so far from reality.

I particularly like the small interactive cards (yes, not really AR) and the various AR (both the tablet sized ones and the one or two HMD-based ones).

Here are some links:

http://www.kzero.co.uk/blog/?p=2715 

http://www.istartedsomething.com/20090228/microsoft-office-labs-vision-2019-video/

I’m a big proponent of handheld augmented reality;  I think that AR is going to be a key technology and conceptual framework for mobile applications.  After all, many of the things we imagine doing on mobile devices relate to the world we are moving through, so AR seems a natural fit.  When practical headmounts finally arrive, things will only become more compelling.

However, the main thing holding AR back, in my opinion, is a clear portfolio of compelling applications, games and uses;  sure, everyone imagines all kinds of things;  applications like Wikitude are evocative of more powerful, practical and useful systems.  

But, frankly, most of what’s been actually deployed has only been noticed because of the GeeWiz factor.  A static model of a car sitting on a paper ad?  Yawn!  Books with 3D graphics that pop out of the page, but don’t really add much to the experience?  Ho hum.  

So why do I think things are about to change?  The FLARtoolkit, that’s why.  Over the years, I’ve worked with designers and other non-technologists because I believe that it will be these sorts of folks, folks who aren’t necessarily in love with the technology but you have something they are trying to do, some vision that the technology enables.  We built our DART tools to let folks work on AR in Macromedia Director.  But, a big limit is that we couldn’t deploy experiences easily;  they still needed to be “installed” to work.  And our camera access and other things were “finicky”. 

But, now, people can build Flash applications that use the camera on your computer to do AR.  Sure, it’s limited;  Flash doesn’t really support 3D, so the amount and kind of content you can create isn’t “professional game quality”;  you can access local devices (like a compass or gps, or more powerful sensors).  

But, you can do simple things.  For example, consider “Eye of Judgement,” the relatively disappointing AR game on the PS3.  All it uses is a stationary camera and markers.  What would the Flash-Games equivalent of that be?  What clever new interactions or game concepts could make such a setup compelling?  Now, we can find out, because now virtually any Flash programmer/author can try it.  Without help from a programmer.  Without installing more software.  With the ability to distribute it!

Consider these demos, here and here.  They aren’t that novel or exciting.  But I could get design and art students to build these.  On their own, without a bunch of technical support.  And that means, very soon, we should start seeing new, exciting things coming out.  And the full source to that first one is available, for anyone who wants to look at it.

Which is cool.  Very cool.

First, in the spirit of full disclosure, I will point out that one of the students in my lab built this demo.  So, I’m obviously biased.  

That said, I am pretty impressed with what she managed to pull off in a very short time.  Ignoring the work involved in hacking the iPhone to get video out of the camera, and porting a little 3D loader/rendering to the phone, she managed to work in quite a few of the ideas we’ve been batting around about how to take advantage of the iPhone for handheld AR;  a bunch of things are still not in there, but will be eventually.  

The idea motivating this project is the old “Virtual Pet”, with the goal of having an individual’s pet be available on different platforms (handheld 3D, handheld AR, desktop 3D, etc), and of having multiple people’s pets be able to play with each other.  More interestingly, we hope to have the environment’s in which you play with your pet affect their growth and behavior.  We’ll see how far we get!

Here’s a video of the current prototype.

YouTube Preview Image

I just saw this post showing a video of an AR demo done in flash.  It’s cool;   I didn’t think Flash was fast enough to run the ARToolkit.