I was going to expand on my predictions that Rouli had posted on Games Alfresco because, frankly, they were pretty lame (mine and not the other nine, those were good.)  But decided that there have been enough predictions for 2010.  So instead, I want to go over the things I want to happen in 2010 in regards to augmented reality.   

1. I want the Nexus One phone from Google to be untethered, cheap and make AR apps fun.

2. I want to be surprised by an AR ready HMD.

3. I want to see fun, creative AR games that are across all platforms and come at an affordable price. 

4. I want the AR inspection assist project I’m working on with Metaio to go flawlessly and for it to revolutionize the way we do difficult inspection job at Toyota and make it easier on our team members. 

5. I want Google Goggles to be a database that other programs can use for pattern recognition and markerless tracking.

6. I want to attend ISMAR10 even though its all the way over in South Korea.

7. I want the ISMAR09 presentations to be put up on YouTube so we can see all the great things that happened.

8. I want Apple to free their live video API for better AR on the iPhone.

9. I want to know what Neogence Enterprises has been working on all these years. 

10. I want to continue to make Games Alfresco the hands-down, defacto source for all your augmented reality news. 

So for all you programmers and entrepreneurs working on the latest in augmented reality tech, even though I may put up your YouTube video or link to your webpage and make semi-snarky comments about its usefulness or how its so-2009, I certainly appreciate your hard work.  Unless you were just mailing it in hoping to capitalize on the AR buzz, then you deserve it and then some.  For all of you in the former category, I leave you with my two favorite quotes to keep you going when things get tough:

All courses of action are risky, so prudence is no in avoiding danger (it’s impossible), but calculating risk and acting decisively.  Make mistakes of ambition and not mistakes of sloth.  Develop the strength to do bold things, not the strength to suffer. 

   — Niccolo Machiavelli, The Prince

IT IS NOT THE CRITIC WHO COUNTS: not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better.  The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood, who strives valiantly, who errs and comes up short again and again…who spends himself in a worthy cause; who, at the best, knows, in the end, the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly. 

   — Teddy Roosevelt

The year 2009 has been an important milestone in the development of augmented reality as an important future technology.  While markers and marketing campaigns captured the early interest of the year, the bigger story was the porting of augmented reality to the various smartphones in the second half. 

But those two generalities aren’t the only thing that happened in 2009 in regards to augmented reality.  I’m going to take a look back at the year, month-by-month, to remember how the year unfolded, so we better understand the direction of the future. 

January

Back at the dawn of 2009, all the augmented reality news you needed to know was in one place – Games Alfresco.  Hope was high and all the pieces seemed to be clicking into place. 

Biggest News of the Month – Metaio releases its Lego AR boxes

   * By a long shot, Metaio’s release of the augmented reality Lego box was the biggest news of the month.  When you can get Angela Merkel and the Govenator to pose with your product, you’re doing well.

Coolest App/Video – iPhone App Helps Solve Rubix Cube

   * This app portents of things to come in the future as products like Google Goggles amp up the possibilities of reality-search.  Using specific algorithms (agents) for specific tasks, there will be an app for everything you want to do in the future.

Best Article / Interview – UgoTrade Interview with Robert Rice

   * Though Robert hasn’t delivered on his promise of a Neogence product by mid-year (though the Apple video API has stymied their progress), the rest of the interview is full of great quotes and nuggets about augmented reality.

   * Notable quotes from Robert that show how much he understands the industry:

But yes, I’d say that the next 18 months are going to be very interesting with a lot of money being thrown around, new ventures, and plenty of content/applications. I expect most of this will be centered on single user AR experienced through a mobile device with a screen (iPhone, android, etc.). I expect that there will be a significant boost after Vuzix releases some of their wearable *transparent* displays, putting Microvision back into the “has potential but is too quiet” position.

Remember, one of the biggest risks that AR has, is being branded as “novelty”, which means “cool for five minutes but ultimately a waste of time.” I think we have a ways to go before something is truly useful, but as 2009 progresses we should start seeing some effort here. I’d guess 2010 before something really useful comes out…at least something practical.

Now, having said that, I should say that I expect entertainment and games to take the lead (as usual), although there are a few companies really trying to leverage AR and video/graphics compositing for marketing (brochures) and location based methods (kiosks, large screen projections, etc.)

Other notable things that happened in January – The Augmented Times is born and Rouli Nir begins a great year of chronicling the happenings of AR. 

An auspicious quote from Rouli:

This blog is about Augmented Reality (AR). It is my firm belief that AR will be the next web revolution (the so called Web 3.0, and forget about that semantic web nonsense). In the next few years, stronger devices and better algorithms will enable us to merge the real world with cyberspace. Using your mobile phone (at first) or head up display (later on) every real object will be augmented and achieve a web presence. Tourism, shopping, advertisement, entertainment and education are only a few areas that will never be the same.

This blog mission is to document this revolution as it happens. We welcome you, and welcome our augmented future.

February

This month is relatively quiet due to it being handicapped by fewer days than the other months. 

Biggest News of the Month – Augmented Reality in Flash Now Commercially Available.

   *  Making the nuts-and-bolts tools of AR available for commercial use is always a good thing, especially when its paired with flash. 

Coolest App / Video – “Sixth Sense” MIT Device.

   * Patty Maes unleashed this phenomenon on the web early last year and it still gets links and mentions today.  While projection based AR is probably not medium that we’ll all eventually use, the video showed tangible use-cases that help the non-initiated understand the embryonic technology.

Best Article / Interview – This comic from the Abstruse Goose sums up a lot about the future of ubiquitous computing (via Augmented Blog.)

Other notable things that happened in February- I started the Future Digital Life blog, which upon reading my early posts, makes me cringe a little.  Oh well, nothing ventured, nothing gained.  Also, Rouli starts the Weekly Linkfest, the best place to find the round up of AR news each week.

March

In like a lion, out like a lamb.  Augmented reality starts to get rolling in March with lots of goodies and promises.

Biggest News of the Month – Metaio and Vuzix Team Up to Create AR Goggles.

   * While the year has been mostly disappointing regarding HMDs, this was the one tangible piece of progress in the bunch.  The video Ori captured at the GDC portends to greater things.  While no companies have released products for the Vuzix+ARCam, we’ve recently seen an impressive project from Craig Kapp with his Whisper Deck. 

Coolest App / Video – Blair MacIntryre’s GA Tech team – Zombie Attack and Pit Strategy.

   * The videos showed what polished AR games can look like. 

Best Article / Interview – I’m going for a two-fer on this one because its hard to choose between Ori’s talk at WARM09 and Robert’s Decade of Ubiquity post. 

   * If you haven’t watched Ori’s talk, I highly recommend it.  Even though I’d seen the video, I made a point of hearing him live at ISMAR09 because I think its such a well thought-out call for human-based technology.  Robert’s vision is a bit more expansive and covers the whole of augmented reality.  I’ll leave you with another Robert quote:

We will break away from the desk, we will throw away our monitors, and our children will laugh at how large our IPhones are. They will struggle with how we ever managed to get work done with “windows” “webpages” and keyboards. They will be unable to fathom the concept of vinyl disks, typewriters, and landlines. But it all starts, and accelerates, during this next decade. Imagine everything that happened in the last decade, and multiply it. You haven’t seen anything yet. The next decade will make the last one pale in comparison.

Other notable things that happened in March – Topps released augmented baseball cards and ISMAR09 begins calling for papers. 

April

Biggest News of the Month- Rouli and Ori team up on Games Alfresco to create the AR hub for all AR related news.   

   * The move allowed Ori to focus more on his company and gave Rouli the helm on the daily AR newscasting.  Looking back at the great job that GA has done, I’d say it was a good move.

Coolest App / Video – Marco Tempest’s AR card trick video.

   * The video hit the web in April and showed us how AR can change our perception of reality. 

Best Article / Interview – Tim O’Reilly talks Web 2.0

   * Augmented Times identified the best quote from the video:

RFID is an evolutionary dead-end … semantic web or RFID is things “wearing name-tags”, and web 2.0 is learning to recognize things … We’re getting to that kind of augmented reality, where our computers will have senses that are as good as ours or better … they are going to recognize faces, they are going to recognize objects, they gonna have immediate recall. If you ask me “what’s the UI in five years”, it’s a pair of glasses … I’m gonna have some kind of little heads up display because I’m gonna look at something, I’m gonna walk around at a meeting and it will go “that’s Joe, you met him three years ago”.

Other notable things that happened in April – Georgia Tech fear of heights video (another peak into the future of augmented reality as a perception changer), Rouli’s call for marketers to stop using AR to sell cars, Lester at the Augmented Planet gets his blog started, Nokia point and find is out and a couple of articles about brain-computer interfaces: Honda controls robot and the brain-twitter breakthrough. 

May

Biggest News of the Month – Metaio releases Unifeye design demo.

Coolest App / Video – Useful AR from the US Postal Service.

   * Using FLARToolkit, the US postal service shows us that not all AR applications have to be novelity ones. 

Best Article / Interview – Tish interviews Ori Inbar about mobile augmented reality. 

Ori: Just like with every emerging technology in history, people never bought the technology, they bought the content, the apps, the benefits that came on top of the technology. Whether it was VHS winning over Beta Max, or BluRay winning over HD. It’s always because of more/better content. Look at the video game console war: Xbox, and Nintendo did better than Sony just because they had more and better games. Even Windows was a success thanks to its applications. People bought it for the applications not the OS. The content is the first to drive demand.

Other notable things that happened in May – My post about the Path to Augmented Vision, Programmer Joe talks AR at LOGIN, and 11 Industries to be Reinvented with Augmented Reality. 

June

Biggest News of the Month – Layar is online. 

   * The biggest news of the month by far.  Layar was the first big product release that got a buzz bump. 

Coolest App / Video – Zugara Social Shopper.

   * The video showcased the use of motion capture to help eliminate the burdensome need for keyboards and mouses when you’re using augmented reality.  Zugara later put the technique into practice for their Cannonballz and Fashionista products.

Best Article / Interview – UgoTrade Interview with Blair MacIntyre

   * No surprise here.  Tish Shute continues to get great interviews with everyone important in the AR field. 

The problem with the mobile phone as a AR device is that problem of awareness. If I have a head mount on and I walk down the street and there is bunch of probably-not-useful-but-potentially-useful information floating by me, that’s a good thing, because I may see something that is useful or makes me think of something else.  But if I have to hold up my phone to see if something might be interesting nearby, I will never hold up my phone because at the time there is a high probability that there won’t be anything particularly important there.  You might imagine you can get around this by using alerts or something like that, but then you overload whatever alert channel you use.

Other notable things that happened in June- Ori asks us all about our favorite AR games of all time, Noah breaks onto the scene with his Touchless Glove Interface, and Goggle presents their paper on Computer Vision and Pattern Recognition (which later becomes Goggles). 

July

Biggest News of the Month – The AR Consortium Announced. 

   * While its still unknown if the consortium has accomplished anything through cooperation, the intention of cooperation is a start.  Here’s to hoping we’ll see more fruit from this vine in 2010.

Coolest App / Video – AcrossAir Tube Finder.

   * At the time it was released we weren’t sure if it was a real product, but eventually we learned the truth and for a time, AcrossAir became the top selling AR app on the iPhone appstore. 

Best Article / Interview – UgoTrade Interview with Ori Inbar “Reality Reinvented”

Ori:You are so right. It’s multiple elements in the industry that have to come together. You have the technology companies like Imagination and Metaio, then you have content companies like what we’re trying to do; the hardware vendors and the large content providers. Those brands that we’re trying to go after and educate them about the potential of AR. All these pieces need to come together for this market to ignite.

Other notable things that happened in July – James Alliban’s business card goes viral and AR Avatar toys are released at Comic Con.

August

Biggest News of the Month – Yelp Introduces AR Monocle via Easter Egg

Coolest App / Video – Bruce Sterling’s Layar Keynote – At the Dawn of the AR Industry

   * Bruce released an AR-infused novel Catyadids in 2009, so it was only natural that Layar asked him to speak about the fledgling industry.  The speech is memorable with too many great lines to pick from for quoting here. 

Best Article / Interview – Thomas Wrobel’s Proposal for an Open AR Network

   * Easily Tish’s best interview because it turned into more than a reflection of past events or hopes for the future.  Tish and Thomas have since rallied AR professionals and amateurs to join the AR Wave collaborative effort.  They seek to use Wave as the platform for an open AR network and have recently been testing code. 

Other notable things that happened in August – AR Strippers!

September

Biggest News of the Month – Vuzix 920Wrap Will Not be See-Through

   * The biggest disappointment of the year was the news from Vuzix.  Good see-through AR glasses are supposed to release us from the tyranny of the “magic lens”.  Hoping 2010 brings better news, but I’m really waiting until 2011. 

Coolest App / Video – Augmenting Aerial Maps with Dynamic Information

Best Article / Interview – Gene Becker’s Design Strategies for Magic Lens.

Other notable things that happened in September – No more Gamaray - Death of a Browser, Int13 and Total Immersion getting into bed, and the augmented reality reading list.

October

Biggest News of the Month – News from ISMAR09, News from ISMAR09 #2 and UgoTrade ISMAR09 Coverage.

   * I was lucky enough to attend and speak at ISMAR09.   The energy of the participants was exceptional and I came away from it more jazzed about AR than I was going in.  Hopefully the ISMAR committee will be releasing the videos from the event soon. 

Coolest App / Video -  2D sketches become 3D reality. 

Best Article / Interview – ISMAR09 HMD Review

   * I think I summed up the current state of HMDs for AR pretty well.  I’m hoping 2010 brings a suprise for AR HMDs.

Other notable things that happened in October – AR Wave discussion on UgoTrade, Wikitude for the iPhone first look and Giant Hand Torments City

November

Biggest News of the Month – Junaio hits the app store. 

Coolest App / Video – Interactive Entertainment Using AR

   * The videos about the Disney experiences were similar to what Mark Mine showed us at ISMAR.  They really show us how technology can transcend our perceptions of reality.

Best Article / Interview –  Augmented Planet’s Head-to-head browser test (and part 2)

   * Lester broke down the various browsers in this excellent hands-on challenge.  Read both parts to find out the winner.

Other notable things that happened in November – Business Week’s Article about: Getting Beyond the Hype and Robert Rice’s reply. 

December

Biggest News of the Month – Goggle Googles is released.

   * The giant has awoken.  Goggles has many implications for Googles ideas on the AR market.  This won’t be the last time we’ll hear from them about this technology.

Coolest App / Video – Layer 3.0. 

   * While Layar has since pulled the app from the app store and had to issue and apology, the version 3.0 still holds lots of promise. 

Best Article / Interview – Augmented Reality in 2010: Predictions (10 part series)

   * A wide swath of the AR blogosphere had the opportunity to participate in the predictions series from Games Alfresco run by Rouli Nir.  More interesting, snarky, and possibly-true predictions that you can shake an augmented stick at.

Other notable things that happened in December – AR Wave FAQ on UgoTrade, Enkin acquired by Google, ARDevCamp, Wikitude Teams Up with Lonely Planet, 10 Worse Uses of AR in 2009, and Nexus One News and Implications of AR. 

What It All Means

Whew.  After reviewing hundreds of articles and videos, I’m sure I’ve missed a number of important happenings from the year 2009.  If I have, please make sure to comment and I’ll add it if I have overlooked it. 

What does it all mean?  That is entirely up to you.  The year has brought many advances to augmented reality and the ceiling is sky-high for 2010.  There will be surprises, excuses, apologies, releases, re-releases and more.  Money will be pumped in, not all where it should go, and progress will be made.  We’ll laugh at the videos, smile at our favorite products and complain when it does work as expected.  Either way, it will be a year more interesting than the last and I’ll be here to watch it all unfold.

I’m not sure how I missed this at ISMAR09, but wow, I’m impressed.  Michael has clearly thought about the user experience and how to make the technology seemless with reality.  I’m not entirely sure the eventual application, but I want to paint my Coralla with AR flame effects with it right now. 

This is the video that accompanied my poster at ISMAR 2009. The system I developed allows a user to digitally airbrush onto physical objects, with the ‘paint’ being projected onto the objects. The user is given a stencil, which can mask areas of the paint. The stencil tool is a board, but the actual shape of the mask is virtualised, which means you can do things like allowing the user to draw their own stencil to accomplish a specific task.

The key point in this video is building the user interface from physical tools that are augmented with digital information. This idea has uses well beyond the scope of a painting application.

 

Augmented reality can help us see back in time, making history come alive.

The presentation at ISMAR from the Beijing Instituite of Technology showed how they could use AR to reconstruct the Yuanmingyuan, or “Garden of All Gardens” without damaging the current appearance of the ruins.  The Yuanmingyuan was burnt down by Anglo-French forces in 1860 and their project uses AR to project the original architecture onto the site. 

 They plan to utilize a coin-operated viewer to allow tourists to see the AR version of the site.

This type of historical eye is also being demonstrated in Cluny France to show what the abby looked like before it’d been destroyed during the French Revolution.   

I predict (since we’re all busy predicting the AR future) that you won’t be able to visit a historical site in five years without an AR viewer to see the past.  Old civil war sites won’t be the same when you can watch a thousand Union troops storm the Confederate lines.  Maybe kids will cheer when dad fires up the old Studebaker for a trip across the country to revisit historical sites. 

But the real question is–will AR be able to help when you have to drive eighty miles out of the way to visit the giant ball of twine?

There’s so much for me to talk about from ISMAR09 and I was only there for half of the conference.  I have a half-dozen more posts sketched out for the next couple of weeks.  I did get to attend the demo night on Monday which showcased the real hands-on applications of augmented reality.  Gail Carmichael posted up a video of some of the demos, so I’ll try to expand on what was shown.

Sony EyePet Demo- Ever since I saw the trailer for this game, I’ve been wanting to own it.  Even so much that I’m willing to buy a PS3.  The ability for the camera to pick up hand motions was impressive.  In the video, he’s bouncing the head of an AR bobble-head doll to make bubbles come out and tickling the monkey with his fingertips.  As a game, its mostly a cute demonstration of the technology that aims at the 3-8 year old market (and AR enthusiasts), but it’s a precursor of bigger things.  In the future, motion capture will be the new controller. 

The Tank and Kid Demo – This one showed how virtual objects and real ones can interact in a seamless manner.  Once again this technology will be best used in games, but it could bleed over into many other applications. 

Shooter VR/AR Demo – Notice I’m not using the real demo names because I’m not even sure what “Computing Alpha Mattes in Real-Time for Noisy Mixed Reality Video Streams” means.   Unfortunately, its hard to get a feel for what this demo did from the video.  The video makes it look like a cross between Max Headroom and a VR game.  In some ways, that’s all it was, because it used blue screen technology to mix in virtual reality dioramas with the player.   I found it interesting when the player would look at the area at the edge of both the real and the virtual.  I got a real sense of how these two realities can mix together at the edges.  Let’s hope they can figure out how to do this without the blue screen. 

ProFORMA Rapid Model Acquisition - Here’s one I can almost understand from the abstract title.  The program creates 3D models in real-time which is mind blowing.  The downside is you need to rotate the object around for the camera to pick up the object, but the usage has crazy possibilities.  It won the Best Demo for a good reason.  Mix the ProFORMA with other technologies like photosynth and we can achieve a 3D mapping of the world in rapid (4-5 years) time.  More on ProFORMA here. 

Animatronic Shader Lamps Avatars - I would have been more impressed by this demo if Mark Mine from the Disney Imagineers hadn’t explained this same technology during his talk.  Regardless, it grabbed attention because they had a comic as the face making fun of passerbys.

Thanks to Gail Carmichael who took the video and also posted more pictures about it on her blog.  I sat next to her during the Disney keynote while she took tons of pictures with her giant expensive looking camera and uploaded them to her Flickrstream.   I had total camera envy and was afraid she’d laugh at my tiny phone camera.  Cheers to you Gail for helping put on a great ISMAR and taking fantastic pictures.

The biggest barrier to true augmented vision is an HMD that can handle all the requirements.  I had the oppertunity to experience what Microvision, Vuzix and ORALab/EvoOpticks had to offer.  I’ll try to relate their talks and my experiences with their products and let you know if any of them have achieved augmented vision. 

Microvision – Ben Averch

Ben started the presentation during the Industrial section of Workshop Monday.  The first AR-style model Microvision tried was the Nomad which tried to be a hands-free automotive maintenance model.  The unit was plagued by poor ergonomics and eye strain.  It only used the red spectrum and they retired the product line in 2006 only two years after release. 

Microvision then changed their plans to achieve an AR enabled HMD by leveraging their military contracts.  The plan is to sell their Urban Warfare model (Ultra-vis) to the government and then use what they’ve learned to create a consumer model. 

They see a big market for a consumer HMD display but need to solve the remaining issues before its ready for the public.  I had an opportunity to try out their prototype HMD.  It wasn’t a full unit, but a bulky monocle on a stick.  While it didn’t give me an idea how it would function as a pair of sunglasses, it did give me a feel for the graphics.  I was impressed by the clarity of the color in the monocle.  The big problem I see for the first generation of HMDs will be the field of view.  This prototype only had a ~25degree field of vision.  The limited view will mean our early HMDs will only mimic our computer screens and not release the full spectrum of hands-free AR. 

Ben has written about his experience on his official Microvision blog and has said they expect a commercial product by 2011. 

It’s clear from the discussions I had with various industry members that a see-through wearable display that meets people’s expectations from both ergonomics and display performance is the big, obvious missing link in the AR solution story. We at Microvision are working hard to fill in this gap and create a technology solution that can allow this new market to take shape.

Ben also showed off their projector which at the time didn’t excite me much from an AR point of view.  However, after the key note speech from Mark Mine from Disnery Imagineering, I learned that the projector had a lot more going for it than I realized. 

Vuzix – Paul Travers

Paul gave us a nice history of HMDs including the failed hype of VR which has made getting money for HMDs harder to come by.  On a good note, Vuzix has a good track record of selling video glasses which gives them a good base to work from once they master the AR HMD.  Currently they’ve sold around 200,000 video glasses by marketing them the same way a HD TV would be sold.  This might mean that augmented reality’s best bet for widespread use is to be a Trojan horse within another accepted technology (similar to how Yelp gained AR eyeballs through their Monocle easter egg.) 

Unfortunately, Paul had to apologize a few times for misleading on the AR community about the “see-through AR HMDs” everyone thought were right around the corner with the 920Wrap.  He hopes they can reach market by 2010, but like Microvision they still have challenges to solve. 

I did get to try their video glasses, which were interesting, but I would have preferred to try out the glasses Ori tried at GDC.   Paul did say that see-through AR glasses would be priced from $199-349.  This sounds like a great price point that would help gain wider audiences.  I know I would buy one. 

ORALab/EvoOpticks

This presentation delved into more historical and technical realms rather than commercial usage.  Not a lot to pass along, but I did get to try their see-through HMD.  The view port was around 20 degrees and seen through two reflected mirrors on the screen rather than a laser created image.  They have no current plans for commercial production.   

And for the history buffs, the first patent for an HMD was by Albert B. Pratt in 1916. 

I wish Lumus had also attended ISMAR, but no such luck. 

Overall, I was disappointed that no HMD maker had a surprise announcement that they had a product ready for release Q1 2010.  Release dates of 2011 sound more like appeasement than real schedules.  Having experienced the products myself, I also realize the field of view will be a problem for early developers.  Don’t get me wrong, I’d be happy to have a HMD right now, even with a 15 degree field, but the small width will limit creativity.  Hopefully when they solve the lighting and weight issues they’ll also figure out how to achieve a 40 degree field of view. 

This past May, I predicted true Augmented Vision would occur around 2015.  After the demonstrations on Monday, I’m not going to change my prediction.  While I heard many quote Moore’s Law in regards to AR, I’m not buying it yet.  Moore’s Law is fed by billions (with B) of dollars which pushes the technology as fast as it can go.  AR is still languishing in VR’s dead space and has to catch a few wins of its own before it can start generating real cash to take advantage of Moore’s Law.   

Even if Microvision or Vuzix or Lumus release a commercial product in 2011, the software and apps to make the experience fulfilling won’t be available.  I also expect the first versions of the AR HMDs will only be bought by the early adopters.  It will take a few tries to make them lightweight, eye-strain free and with enough field of vision to keep people from having to rotate their head constantly.  I’m all for being wrong on this one, but I expect the road to widespread acceptance will be paved with a few missteps.  Cellphones took twenty years to go from the bag phone to the ultra-sleek smartphone, and while technology moves faster these days, I’m still sticking with my year 2015 prediction.

I’m busy with last second preperations for ISMAR09, but wanted to pass along this video from the IDXA event on Tuesday.  The video shows a variety of augmented reality demonstrations from Bruno Uzzan the CEO at Total Immersion.