Bruce Sterling is wrong about Augmented Reality.

First off, I want to express that I have the utmost respect for Bruce Sterling as both a writer and a visionary.  As a longtime fan of cyberpunk and science fiction in general, Bruce has been a part of the pantheon of authors I regularly visit.  And as a writer, I study his stories so I can improve my practice of the craft.  Just last week I was reading his story “Our Neural Chernobyl” in the teaching anthology Paragons and was blown away by his ability to tell a thrilling story devoid of characters and plot and subsisting entirely of theme.  Bruce is a true master of the craft of writing.

But on the subject of augmented reality being a subject of literary endeavours, I believe, and will attempt to prove in this article, that Bruce Sterling is wrong.

Last week, in an always entertaining interview from Tish Shute and Ori Inbar on UgoTrade, Bruce Sterling stated that: “I’m not sure it makes a lot of sense to write fiction nowadays “about AR,” because it’s no longer a fictional topic. It’s become like writing fiction “about cinema.” You can write good fiction about someone who works in cinema, but not fiction about cinema itself. AR is not sci-fi “Augmented Reality” any more, it’s become a real-world phenomenon, a new industry of real augmentation..”

As I understand it, his basic argument is that because augmented reality is a real technology now and not just a fictional – though probable – idea, that it is not a worthy subject for writing fiction about.  And when we say “about,” we can probably assume that Bruce doesn’t just mean that AR cannot be a part of a story.  More so that the story cannot be an exploration of the idea of augmented reality, because the idea already exists as a tangible product that one purchases and uses and therefore there is nothing to explore when anyone can visit.

Or said another way, science fiction is about exploring the possibilities presented by technology in relation to the human condition.  If that possibility has become an actuality, then what is left to explore?

So there’s where I will begin my counter-argument.  With the assertion that augmented reality has become, excuse the redundancy, a reality.  What we’re seeing currently in the marketplace and even in the research domains are nascent attempts at creating the experience of augmented reality.

Often when we’re speaking of the true potentials of augmented reality, we’re using words like immersive and ubiquitous.  The artifices are data shades and eye-screens in the form of contacts.  The data stream is superfluous and overwhelming.  We have none of these things currently.

What we are seeing in the marketplace and in the research labs are gimmicks and toys and games.  We have many examples of AR that stir the imagination.  I, for one, have been promoting these applications of AR for many years now.  But is AR a technology that has truly changed the way we work and play and live our lives?   Not at all.  The yellow line in an NFL game is the most common way that AR has touched the masses.

And once a technology becomes reality, why should science fiction cease to care about it?  I speak not about technology as a prop or the furniture of the story (to use a term by George RR Martin), but to write about the story as it revolves around the technology, changing people’s lives by their use of it and thus showing the technology through the lens of human behavior.

I also believe that AR is still a relevant focus for science fiction because AR is not a simple tool, it is a whole new medium for the transfer and dissemination of information.

The nearest and probably most obvious comparison is virtual reality.  VR at its best was a destination and a substituite for the real world.  VR has always been a difficult sell to the masses and a narrow niche best populated by the niche groups that could take advantage of its morphology with the best example being the furries in Second Life.

AR differs from VR in the same way that the national highway system differs from a late-night Denny’s.  AR, as a new medium of information transfer, will change our daily lives, while VR was a place to escape from reality.

I’m not suggesting here, that Bruce was arguing that AR and VR are the same.  But I think illustrating the differences helps explain why AR is a new medium rather than just a new technology that will change people’s lives in varied and profound ways, so it cannot be dismissed as a topic for fiction just because I can swat invisible mosquitoes on my iPhone.  We’ve barely scratched the surface (or multi-touched it.)

The magic of humanity’s rise through the evolutionary brackets has been from its superior ability to communicate information.  The greatest of game changing technologies have always been the ones related to medium: spoken language, written language, printing press, radio, television, the Internet and now augmented reality.

The wonder of AR is that we’re taking these high powered computers we carry in our pockets and the gargantuan mountains of information stored on clouds and hard drives and attaching them to the moments and locations we need them.

As a fifteen year employee of Toyota, I can assure you the key ingredient to the Toyota Production System that has changed industries worldwide, is the ability to get the right information to the right people at the right time.  It all comes down to that.

And that’s a major reason why I believe that AR is still ripe as a topic for science fiction.  As a medium for information, it can be used as more than just a travelling on-the-spot wikipedia.  Augmented reality invites both the viewing, but the creation and collecting of new information.

AR is also paradoxically about the efficiency of human action in relation to usable data and the avoidance of reality in the form of pictures and graphics.  AR is more than just data to find the best routes to the local Starbucks.  It’s also seeing your local street corner as a property in a city wide game of Monopoly.

The best mediums, like television and the Internet, both instruct and destroy.  Information can be additive and in the same breath addictive.  If only Philip K. Dick were alive, he would have a lot to say about AR (and in some ways, he already has.)

Lastly, I say that AR can be a topic of science fiction as a writer–though I am no where near the stature of Bruce Sterling, nor have even the slightest twinkle of his legacy.  Instead, I offer my own experiences as proof and that I am interested in AR so much that I’ve written three books, have another three in the planning, and am producing an anthology this summer–all about AR.  I’m putting my money where my mouth is, so to speak.

And Bruce might respond that the ability to do something is not the same as that it should be done.

Which is a valid point.  How can one decide if AR is still worthy to be written about?  Is it based on awards?  Vernor Vinge won the Hugo for Rainbows End, the only book about AR that I’m aware of besides my own.  But that was four years ago in 2007.  Have any other books about AR even been written in that time?

A couple of years ago, I collected with Bruce’s help, the totality of writings about AR at the time.  I’m sure we missed some stories and books, but most of them used AR in only a tangential way–more of a neat toy or a furniture, rather than exploring how AR changed people’s lives.

Then should sales be the deciding factor on if AR should be written about?  If readers aren’t interested in purchasing it, then its time has come and gone.  But once again, we’re presented with too little data.  Even myself, at this time, have only one indie published book about the subject and too little sales data to mean anything.

And maybe that’s why it cannot be said that AR as a topic for science fiction has already played out.  We don’t know yet as there isn’t enough data.  As a logical thinker and a visionary, I think Bruce would respect that.  In fact, Bruce states in this article that: “any good futurist is a historian.”  We have so little history on AR as an impact to human life.

I also suggest that we shouldn’t be in a practice of talking about things that shouldn’t done (minus the obvious ethical and moral cases.)  Who’s to say that every nook and cranny of any science-fiction topic has been thoroughly explored? Even well-known cave systems have uncovered new sections long after the “experts” have marked it off the list.

So I say that augmented reality should be a topic for science fiction writing for years to come, and not just as an about, but also as a backdrop for other stories.  The medium is just too rich and varied to be ignored.

Last Note – I considered whether or not I should post this for a few days, deciding if I had written it for conflict’s sake, or if I truly had a constructive argument.  My conclusion, as you can obviously see, was to post.  Conflict for the sake of conflict is pointless and serves only the look-at-me type of blog poster sometimes found on the Internet today.  Constructive conflict that instructs the arguer and hopefully the reader, and maybe even convinces the counter-arguer to reconsider their point, is worth the conflict.  And since I felt I learned from writing this piece, in the end I should share it.  And besides, writing posts that no one reads, even if they are a constructive argument, is like arguing alone and in the dark.  Sure, you’re arguing, but in the end, you’re still alone and in the dark.

…but not here.

Trying a little something different with some of my augmented reality posts.  Going to just single post some of them over on Games Alfresco, especially the video heavy ones like this.  With a little help from Ori (well, a LOT of help), I put up a listing of many of the current AR games out there.  I’m sure I missed some.  If I have, post ‘em in the comments section over there.

Now this doesn’t mean I’m not going to talk about augmented reality over here on TFDL.  But I’ll keep my general musings on AR to this channel and keep the more specific business and products to Games Alfresco.  I reserve the right to change my mind though.

“It’s better than Angry Birds,” said my nine-year old daughter.

And who am I to argue about that, but given that Angry Birds has dominated our house for the last two months, I was just a little surprised.

Now the first question, one might ask is: “Why are you letting your daughter play Angry Birds all the time instead of doing school work.”  Fine, good question.  But I would answer: “Because she gets her school work done quickly and because it’s actually her iPod Touch that Angry Birds lives on, and if I’m not nice to her, then I don’t get to play.”

See.  A simple answer.

Now the second question would be: “Why would she like Paranormal more than Angry Birds?!  Everyone loves Angry Birds!  It practically comes with any smartphone now.  Is there something wrong with her?”

Well, yes there is.  She’s my daughter, first off, and so that puts her behind the eight ball already.  But the real answer is that she has imagination.

I finally got around to loading up Paranormal on my wife’s iPhone last week.  Yes, I have to be nice to her too, or I don’t get to play with her iPhone either.  Strange how they’ve got me cornered like that.  But I digress.

The game loaded quickly and after finding how to turn the music off (fun for about one minute, then annoying) we started playing it.  The game really comes in three modes: investigations, spells and quests.  Wait.  I’m getting ahead of myself.  First, I should start with what the game is about.

The game involves being a supernatural bad-ass with the ability to destroy ghosts, cast spells, and cleanse the land of evil spirits.  I’m not sure what the official line on the game is since it is a movie tie-in, but that’s what we decided at the ol’ Carpenter house hold.

Through out the game, you gain experience, items, and sanity (it is a fictional game after all) for performing the above listed actions (questing, investigations and spells).  The kids immediately took to the investigations, since you can do it right then and there, spinning around with your “camera” and finding ghosts in the room you’re in.  You get more experience for faster and more accurate captures.

Actions in the game are regulated by mana.  Investigations and spells require mana, which regenerates over time.  You can purchase more mana at the store if necessary.  We didn’t need to.

The second part of the game is casting spells.  The hell holes in your area must be cleansed and kept down through spells.  Utilizing a drawing of a pentagram (very cool I might add, and would make my old elementary Catholic school nuns faint at the thought), the spell can be boosted.  This part of the game is a basic strategy game and involves moving around outside of the home to accomplish.

The last part of the game, the part the kids found to be the most fun (though they enjoyed it all), was the questing.  On the top down map, pins appear showing locations of quests.  Quests must be accomplished at the general area of the pin, so some outside adventuring must occur.  The quests don’t involve much beyond going to the spot, a point that could use some improvement in future games like this, but they did provide story items that helped make the experience spooky.  It also helped, as we were wandering around the neighborhood at 9pm, creeping near a quest point, that an owl hooted scaring the b’jesus out of us.

Overall, I give it two big thumbs up and so do the kids.  Ori Inbar promised the world that he would make games that required our children to get away from screens and adventure into the real world.   That’s the game that his company Ogemento delivered.  While there is some “screen time,” once we got outside, they barely looked at the iPhone and were busy filling in the rest of the game with their imagination.  Another plus is that, while the game uses augmented reality, I never really noticed the technology.  The game was more “experience” rather than “gimmick”.

And though I regularly write for Games Alfresco, which is Ori’s blog; I do not receive any compensation for my articles and most importantly, this article.   The app is free, anyway, so there’s nothing to be gained.  So phhtttt…, if you thought otherwise.

Two years ago, in 2009, augmented reality took its first steps into mainstream consciousness.  This last year, 2010, we’ve seen augmented reality really expand into new formats and across spectra.

Like I did last year, I’m going to review month-by-month the mishaps and happenings of augmented reality in 2010.  This year the task seems more imposing as there’s a lot more content to cover, even though interest on the technology has plateaued.  And I expect that progress has been exciting, we still haven’t hit the real possibilities that AR has to offer and don’t expect to for another few years.

January

Already I’m overwhelmed by the amount of available content to choose from for this review.  Even just a year ago, I’m surprised by how much was going on in the month of January.  Also, the tone of the articles are much different than they are now.  People were still getting use to the idea of augmented reality and not quite sure where the technology was headed.  There’s a little more settling in the marketplace and augmented reality isn’t such a foreign phrase now.

Biggest News – The AR Consortium announces ARE2010.  Reading that article makes me wonder if there’s still anything going on with the consortium or has it just become the way to organize the ARE events.

Coolest App / Video – At this point, apps are still far and few between so the early months will be more devoted to the videos.  This video should of no surprise because it garnered a quarter of a million hits.

The latter half of the 20th century saw the built environment merged with media space, and architecture taking on new roles related to branding, image and consumerism. Augmented reality may recontextualise the functions of consumerism and architecture, and change in the way in which we operate within it.

A film produced for my final year Masters in Architecture, part of a larger project about the social and architectural consequences of new media and augmented reality.

Best Article / Interview – Augmented Reality: Pure Hype or The Next Big Thing by Computer World did a nice job of summing up the technology at that point in time.  Additionally, it gave a good basic lesson on AR to those who’d never heard of it before.

Notable Quotes – This one comes from Tish Shute’s interview with Anselm Hook.

When you are mediating the translation layer between the image and the data, then there is an opportunity for you to control it, and that opportunity is hard to resist.  It is hard to choose not to own that opportunity. It is an advertising opportunity. It is a revenue opportunity. It is a chance to send a message and a tone.

I know that Google and companies like that are keenly aware of the kinds of roles they don’t want to hold, but it is sometimes seductive to think about them. And I am afraid that we, as a community, need to assert an ownership, kind of a commons, over how computers will translate what they see to information that we perceive.

Other Notables – Augmented Planet argued the Case Against Augmented Reality, which garnered some philosophical interest but didn’t really lay out a concrete case against it.  Also, AR Drone.

February

Biggest News – Layar opens their App Store was pretty big news.  Also, they picked up another $3.4M in funding.

Coolest App / Video – The Recognizr became an actual app which opened up the idea that Your Face is A Social Business Card.  No one has come up to me holding up their smartphone to find out who I am yet, but I suppose it’s still possible.

Best article / interview – Tish does all the best interviews.  Here’s another with Brady Forrest: The Physical World Becomes a Software Construct.

Other notables - The Augmented Planet reviews best practices for AR apps, 7 Ways AR Will Change Your Brain, and Augmented Maps with Photosynth.

March

Biggest News – Double shot of Metaio here with the Junaio $5,000 app contest and Junaio 2.0 hitting the iPhone app store.

Coolest App / Video – The augmented reality sketching games really captivated a lot of people by ditching markers and letting people create on the fly.

Best Article / Interview – Visual Search, Augmented Reality, and Physical Hyperlinks for Playfulness, Not Just Purchases: Talking with Paige Saez about ImageWiki.  Does Tish ever use a short interview title?  Geez.

It sucks that it is a closed proprietary system, but the App Store has done something for software that nothing has ever done in the whole world. Software is candy now. It’s candy. It is like when you are waiting at the grocery store at the checkout line and you are stuck behind somebody, and you have got all these little tchotchka’s, candy bars, magazines, nail-clippers and things. That is the equivalent of software now. It’s become an impulse buy, which is amazing. Nobody would ever have thought…that is actually revolutionary. That’s huge.

Other notables – This article of mine hit a StumbleUpon feedback loop and blew the doors off any of my previous articles or since – Eye Tracking Will Be The New Click-Throughs;  another one of note is Total Immersion on Sky News.

April

Biggest News – TechRadar reported that Google Goggles would be opened up to third party apps.  This was big news back then but we haven’t seen much from it.

Coolest App / Video – Augmented driving on your iPhone – the precursor to cars driving themselves?

Best Interview / Article – Has Augmented Reality Peaked? Rouli Nir asks an important question that is still relevant today and so is the answer.

Other notables – AR Wave at Where 2.0 and The Stark HUD 2020 (fake, but makes me drool).

May

Biggest News – Ogmento recieves $3.5M in funding as reported by RWW.

Coolest Apps – Two apps stick out to me this month with Gigaputt and ShootAR.

Coolest Video – What’s another year in AR without some magic from our favorite magician Marco Tempest?

Best Article / Interview – Bloomberg TV’s 30 minute show about augmented reality.   I swear I’m not partial to this video because during the interview portion with Tish (nice to see her on the other side of the interview table), she’s got her computer open to my website.  Besides 30 minutes of AR goodness, this show is a reminder that AR has moved well beyond the university level.

Other Notables – Sixth Sense for Autism and 10 Cool Things Happening Right Now in Augmented Reality.

June

Biggest News – Obviously the biggest news in June was ARE2010.

Coolest App / Video – A three-pack from the event: Bruce Sterling, Will Wright, and Jesse Schell.

Best Article / Interview – More from ARE2010 with Tish’s interview of Bruce Sterling.

Well, it’s because they are trying to do a different thing than media does.  I mean, they are trying to “augment reality.”  They don’t want you to know that you are using a medium.  They don’t want you to realize that you’re watching computer animation overlaid on some video acquisition stream.  That would defeat the whole point of AR.  It’s entirely different from an analog medium like television, where you turn on the television and there’s a constant stream of station identification alerts.   That’s like: “Don’t touch that dial!  You’re on channel 13!  Stay with us!”  Then it’s like, “And now a few words from our friendly sponsors!”  That medium was engineered to keep your eyeballs locked to a single stream that they’re feeding you.

Other notable – Digital Bullets: Is AR a fad or commercially viable.  An interview with various CEOs.

July

Biggest News – Qualcomm’s 2010 Augmented Reality Developer Challenge.  With $200,000 in prize money at stake for this contest, you can be sure there will be some impressive applications when the results are announced on Feb 14th of 2011.

Coolest App / Video – The augmented reality BP logo hack for the “Leak In Your Hometown” brought forth some interesting questions about trademarks in an AR world.

Best Article / Interview – A Short Interview: Celebrating Layar’s First Year from Rouli.

Other notables – The Great Civil War Project.

August

Biggest News – Kooba receives $3.0M in funding.

Coolest App / Video – Two times the dystopic vision.

Best Interview / Article – Lester gives his view that AR glasses are at least twenty years away.  I suspect he’s wrong based on some quick Moore’s Law-like math, but we’ll see.

Other notables – Vision Based AR from Jay Wright (Qualcomm) and 3 Reasons Why Games Are The Killer App for AR from Ori Inbar.

September

Biggest News – The world’s first augmented reality stamp (okay, so September wasn’t a big month for AR news).

Coolest App / Video – Hands On With the Lumus See-Through Wearable Display.  Unfortunately the demo is their helmet version and not a commercial display, but in a year lacking in AR glasses news, I was desperate.

Best Interview / Article – Urban Games, Storytelling with Augmented Reality, The “Big ARNY”, and “Inside AR: Talking with Thomas Alt, Metaio.”

October

Biggest News – Tonchidot lands $12M in second round funding as reported by Games Beat.  As the year goes on, so does the size of funding.

Coolest App / Video - The Sex Offender Augmented Reality Tracker App.  I picked this one for many reasons even though it’s not a great use of AR.  The first is the strange mashup of the viral video guy Antoine Dodson with a serious application.  The second is the implications of the use of data.

Best Article / Interview – Is AR Ready For the Trough of Disillusionment by Chris Arkenberg and Mobile AR is Sexy But Is It Practical by Jack Benoff.

Other notable – The Missing Manual on the Future: Tim O’Reilly’s Four Cylinder Innovation Engine.  It’s not specifically about AR, but it’s an important read.  Google Goggles now available on iPhone ( I probably think this is more important than it truly is since we haven’t seen much from it).

November

Biggest News – Kinect.  Kinect.  Kinect.  While Layar raising $14M in funding was nothing to sneeze at, the buzz has certainly been about the Kinect sensor.  See the Kinect Hacks site for all the best Kinect news.

Coolest App / Video – AirRaid Social Augmented Reality Game brought us a tower defense game combined with AR and social networking.

Best Interview / Article – Google Goggles Tests Ads Triggered By Your Mobile Camera by TechCrunch.  This article is important because it portents the future of advertising.

December

Biggest News – Android 3.0 may have a Google built AR app that comes with the OS?

Coolest App / Video – WordLens has garnered almost 3 million views on its video.  That easily gives it the top spot in our apps category for the year.

Best Article / Interview – Most AR Companies Not Doing AR? from Venture Beat.  I’m not a hardcore definition guy when it comes to AR.  IMHO, if it meets the customers needs, then you can call it whatever you want.

Other notables -10 Awesome Ways To Use Kinect For Augmented Reality,  AR Tags The Sign Of Apps To Come, and Metaio’s AR Magic Tricks.

What It All Means

I’m absolutely certain I’ve missed a few important items from the year or some just never actually came to fruition (hello Mirascape.)  So if I have, let me know in the comments section and I’ll add it to the list.  Some of the themes I notices as I reviewed hundreds of articles was the lackluster state of AR apps.  While we have tons of AR apps now, most of them, minus notables like WordLens, are pretty blasé.

The other common theme was AR companies receiving significant funding.  The total was around $50M between the various announcements, but I’m certain next year will outclass that number by a couple of hundred million.  Given that investors are bullish on AR, we can feel comfortable that we’ve only seen the beginning.

Did the year 2010 live up to your expectation?  It did for me, barely.  While the software and hardware (Kinect!) made advances, I was thoroughly disappointed by the lack of any news on the AR glasses front.  I even tried to put together a “State of AR Glasses” post, but found a couple of the players avoiding my phone calls and emails.  Oh well, if they’re not ready, they’re not ready, I’m hoping next year is better on that front.

But overall, I’m pleased with the direction the technology is going.  Heck, I even published a book and a few short stories about the fictional use of the technology and will publishing a few more next year, so you can be sure I’m bullish on the technology too.   So here’s to another year behind us and a bright shiny new year ahead, complete with surprises, apologies, and new directions we never even thought of.  Either way, I’ll be here to watch it unfold.

Have you ever heard of this guy? A brief interview from Geoweb Forum with Ori Inbar.  In it he talks about ARE2010, ARNY, the state of the industry and a gives a brief demonstration of Put A Spell.

The year 2009 has been an important milestone in the development of augmented reality as an important future technology.  While markers and marketing campaigns captured the early interest of the year, the bigger story was the porting of augmented reality to the various smartphones in the second half. 

But those two generalities aren’t the only thing that happened in 2009 in regards to augmented reality.  I’m going to take a look back at the year, month-by-month, to remember how the year unfolded, so we better understand the direction of the future. 

January

Back at the dawn of 2009, all the augmented reality news you needed to know was in one place – Games Alfresco.  Hope was high and all the pieces seemed to be clicking into place. 

Biggest News of the Month – Metaio releases its Lego AR boxes

   * By a long shot, Metaio’s release of the augmented reality Lego box was the biggest news of the month.  When you can get Angela Merkel and the Govenator to pose with your product, you’re doing well.

Coolest App/Video – iPhone App Helps Solve Rubix Cube

   * This app portents of things to come in the future as products like Google Goggles amp up the possibilities of reality-search.  Using specific algorithms (agents) for specific tasks, there will be an app for everything you want to do in the future.

Best Article / Interview – UgoTrade Interview with Robert Rice

   * Though Robert hasn’t delivered on his promise of a Neogence product by mid-year (though the Apple video API has stymied their progress), the rest of the interview is full of great quotes and nuggets about augmented reality.

   * Notable quotes from Robert that show how much he understands the industry:

But yes, I’d say that the next 18 months are going to be very interesting with a lot of money being thrown around, new ventures, and plenty of content/applications. I expect most of this will be centered on single user AR experienced through a mobile device with a screen (iPhone, android, etc.). I expect that there will be a significant boost after Vuzix releases some of their wearable *transparent* displays, putting Microvision back into the “has potential but is too quiet” position.

Remember, one of the biggest risks that AR has, is being branded as “novelty”, which means “cool for five minutes but ultimately a waste of time.” I think we have a ways to go before something is truly useful, but as 2009 progresses we should start seeing some effort here. I’d guess 2010 before something really useful comes out…at least something practical.

Now, having said that, I should say that I expect entertainment and games to take the lead (as usual), although there are a few companies really trying to leverage AR and video/graphics compositing for marketing (brochures) and location based methods (kiosks, large screen projections, etc.)

Other notable things that happened in January – The Augmented Times is born and Rouli Nir begins a great year of chronicling the happenings of AR. 

An auspicious quote from Rouli:

This blog is about Augmented Reality (AR). It is my firm belief that AR will be the next web revolution (the so called Web 3.0, and forget about that semantic web nonsense). In the next few years, stronger devices and better algorithms will enable us to merge the real world with cyberspace. Using your mobile phone (at first) or head up display (later on) every real object will be augmented and achieve a web presence. Tourism, shopping, advertisement, entertainment and education are only a few areas that will never be the same.

This blog mission is to document this revolution as it happens. We welcome you, and welcome our augmented future.

February

This month is relatively quiet due to it being handicapped by fewer days than the other months. 

Biggest News of the Month – Augmented Reality in Flash Now Commercially Available.

   *  Making the nuts-and-bolts tools of AR available for commercial use is always a good thing, especially when its paired with flash. 

Coolest App / Video – “Sixth Sense” MIT Device.

   * Patty Maes unleashed this phenomenon on the web early last year and it still gets links and mentions today.  While projection based AR is probably not medium that we’ll all eventually use, the video showed tangible use-cases that help the non-initiated understand the embryonic technology.

Best Article / Interview – This comic from the Abstruse Goose sums up a lot about the future of ubiquitous computing (via Augmented Blog.)

Other notable things that happened in February- I started the Future Digital Life blog, which upon reading my early posts, makes me cringe a little.  Oh well, nothing ventured, nothing gained.  Also, Rouli starts the Weekly Linkfest, the best place to find the round up of AR news each week.

March

In like a lion, out like a lamb.  Augmented reality starts to get rolling in March with lots of goodies and promises.

Biggest News of the Month – Metaio and Vuzix Team Up to Create AR Goggles.

   * While the year has been mostly disappointing regarding HMDs, this was the one tangible piece of progress in the bunch.  The video Ori captured at the GDC portends to greater things.  While no companies have released products for the Vuzix+ARCam, we’ve recently seen an impressive project from Craig Kapp with his Whisper Deck. 

Coolest App / Video – Blair MacIntryre’s GA Tech team – Zombie Attack and Pit Strategy.

   * The videos showed what polished AR games can look like. 

Best Article / Interview – I’m going for a two-fer on this one because its hard to choose between Ori’s talk at WARM09 and Robert’s Decade of Ubiquity post. 

   * If you haven’t watched Ori’s talk, I highly recommend it.  Even though I’d seen the video, I made a point of hearing him live at ISMAR09 because I think its such a well thought-out call for human-based technology.  Robert’s vision is a bit more expansive and covers the whole of augmented reality.  I’ll leave you with another Robert quote:

We will break away from the desk, we will throw away our monitors, and our children will laugh at how large our IPhones are. They will struggle with how we ever managed to get work done with “windows” “webpages” and keyboards. They will be unable to fathom the concept of vinyl disks, typewriters, and landlines. But it all starts, and accelerates, during this next decade. Imagine everything that happened in the last decade, and multiply it. You haven’t seen anything yet. The next decade will make the last one pale in comparison.

Other notable things that happened in March – Topps released augmented baseball cards and ISMAR09 begins calling for papers. 

April

Biggest News of the Month- Rouli and Ori team up on Games Alfresco to create the AR hub for all AR related news.   

   * The move allowed Ori to focus more on his company and gave Rouli the helm on the daily AR newscasting.  Looking back at the great job that GA has done, I’d say it was a good move.

Coolest App / Video – Marco Tempest’s AR card trick video.

   * The video hit the web in April and showed us how AR can change our perception of reality. 

Best Article / Interview – Tim O’Reilly talks Web 2.0

   * Augmented Times identified the best quote from the video:

RFID is an evolutionary dead-end … semantic web or RFID is things “wearing name-tags”, and web 2.0 is learning to recognize things … We’re getting to that kind of augmented reality, where our computers will have senses that are as good as ours or better … they are going to recognize faces, they are going to recognize objects, they gonna have immediate recall. If you ask me “what’s the UI in five years”, it’s a pair of glasses … I’m gonna have some kind of little heads up display because I’m gonna look at something, I’m gonna walk around at a meeting and it will go “that’s Joe, you met him three years ago”.

Other notable things that happened in April – Georgia Tech fear of heights video (another peak into the future of augmented reality as a perception changer), Rouli’s call for marketers to stop using AR to sell cars, Lester at the Augmented Planet gets his blog started, Nokia point and find is out and a couple of articles about brain-computer interfaces: Honda controls robot and the brain-twitter breakthrough. 

May

Biggest News of the Month – Metaio releases Unifeye design demo.

Coolest App / Video – Useful AR from the US Postal Service.

   * Using FLARToolkit, the US postal service shows us that not all AR applications have to be novelity ones. 

Best Article / Interview – Tish interviews Ori Inbar about mobile augmented reality. 

Ori: Just like with every emerging technology in history, people never bought the technology, they bought the content, the apps, the benefits that came on top of the technology. Whether it was VHS winning over Beta Max, or BluRay winning over HD. It’s always because of more/better content. Look at the video game console war: Xbox, and Nintendo did better than Sony just because they had more and better games. Even Windows was a success thanks to its applications. People bought it for the applications not the OS. The content is the first to drive demand.

Other notable things that happened in May – My post about the Path to Augmented Vision, Programmer Joe talks AR at LOGIN, and 11 Industries to be Reinvented with Augmented Reality. 

June

Biggest News of the Month – Layar is online. 

   * The biggest news of the month by far.  Layar was the first big product release that got a buzz bump. 

Coolest App / Video – Zugara Social Shopper.

   * The video showcased the use of motion capture to help eliminate the burdensome need for keyboards and mouses when you’re using augmented reality.  Zugara later put the technique into practice for their Cannonballz and Fashionista products.

Best Article / Interview – UgoTrade Interview with Blair MacIntyre

   * No surprise here.  Tish Shute continues to get great interviews with everyone important in the AR field. 

The problem with the mobile phone as a AR device is that problem of awareness. If I have a head mount on and I walk down the street and there is bunch of probably-not-useful-but-potentially-useful information floating by me, that’s a good thing, because I may see something that is useful or makes me think of something else.  But if I have to hold up my phone to see if something might be interesting nearby, I will never hold up my phone because at the time there is a high probability that there won’t be anything particularly important there.  You might imagine you can get around this by using alerts or something like that, but then you overload whatever alert channel you use.

Other notable things that happened in June- Ori asks us all about our favorite AR games of all time, Noah breaks onto the scene with his Touchless Glove Interface, and Goggle presents their paper on Computer Vision and Pattern Recognition (which later becomes Goggles). 

July

Biggest News of the Month – The AR Consortium Announced. 

   * While its still unknown if the consortium has accomplished anything through cooperation, the intention of cooperation is a start.  Here’s to hoping we’ll see more fruit from this vine in 2010.

Coolest App / Video – AcrossAir Tube Finder.

   * At the time it was released we weren’t sure if it was a real product, but eventually we learned the truth and for a time, AcrossAir became the top selling AR app on the iPhone appstore. 

Best Article / Interview – UgoTrade Interview with Ori Inbar “Reality Reinvented”

Ori:You are so right. It’s multiple elements in the industry that have to come together. You have the technology companies like Imagination and Metaio, then you have content companies like what we’re trying to do; the hardware vendors and the large content providers. Those brands that we’re trying to go after and educate them about the potential of AR. All these pieces need to come together for this market to ignite.

Other notable things that happened in July – James Alliban’s business card goes viral and AR Avatar toys are released at Comic Con.

August

Biggest News of the Month – Yelp Introduces AR Monocle via Easter Egg

Coolest App / Video – Bruce Sterling’s Layar Keynote – At the Dawn of the AR Industry

   * Bruce released an AR-infused novel Catyadids in 2009, so it was only natural that Layar asked him to speak about the fledgling industry.  The speech is memorable with too many great lines to pick from for quoting here. 

Best Article / Interview – Thomas Wrobel’s Proposal for an Open AR Network

   * Easily Tish’s best interview because it turned into more than a reflection of past events or hopes for the future.  Tish and Thomas have since rallied AR professionals and amateurs to join the AR Wave collaborative effort.  They seek to use Wave as the platform for an open AR network and have recently been testing code. 

Other notable things that happened in August – AR Strippers!

September

Biggest News of the Month – Vuzix 920Wrap Will Not be See-Through

   * The biggest disappointment of the year was the news from Vuzix.  Good see-through AR glasses are supposed to release us from the tyranny of the “magic lens”.  Hoping 2010 brings better news, but I’m really waiting until 2011. 

Coolest App / Video – Augmenting Aerial Maps with Dynamic Information

Best Article / Interview – Gene Becker’s Design Strategies for Magic Lens.

Other notable things that happened in September – No more Gamaray - Death of a Browser, Int13 and Total Immersion getting into bed, and the augmented reality reading list.

October

Biggest News of the Month – News from ISMAR09, News from ISMAR09 #2 and UgoTrade ISMAR09 Coverage.

   * I was lucky enough to attend and speak at ISMAR09.   The energy of the participants was exceptional and I came away from it more jazzed about AR than I was going in.  Hopefully the ISMAR committee will be releasing the videos from the event soon. 

Coolest App / Video -  2D sketches become 3D reality. 

Best Article / Interview – ISMAR09 HMD Review

   * I think I summed up the current state of HMDs for AR pretty well.  I’m hoping 2010 brings a suprise for AR HMDs.

Other notable things that happened in October – AR Wave discussion on UgoTrade, Wikitude for the iPhone first look and Giant Hand Torments City

November

Biggest News of the Month – Junaio hits the app store. 

Coolest App / Video – Interactive Entertainment Using AR

   * The videos about the Disney experiences were similar to what Mark Mine showed us at ISMAR.  They really show us how technology can transcend our perceptions of reality.

Best Article / Interview –  Augmented Planet’s Head-to-head browser test (and part 2)

   * Lester broke down the various browsers in this excellent hands-on challenge.  Read both parts to find out the winner.

Other notable things that happened in November – Business Week’s Article about: Getting Beyond the Hype and Robert Rice’s reply. 

December

Biggest News of the Month – Goggle Googles is released.

   * The giant has awoken.  Goggles has many implications for Googles ideas on the AR market.  This won’t be the last time we’ll hear from them about this technology.

Coolest App / Video – Layer 3.0. 

   * While Layar has since pulled the app from the app store and had to issue and apology, the version 3.0 still holds lots of promise. 

Best Article / Interview – Augmented Reality in 2010: Predictions (10 part series)

   * A wide swath of the AR blogosphere had the opportunity to participate in the predictions series from Games Alfresco run by Rouli Nir.  More interesting, snarky, and possibly-true predictions that you can shake an augmented stick at.

Other notable things that happened in December – AR Wave FAQ on UgoTrade, Enkin acquired by Google, ARDevCamp, Wikitude Teams Up with Lonely Planet, 10 Worse Uses of AR in 2009, and Nexus One News and Implications of AR. 

What It All Means

Whew.  After reviewing hundreds of articles and videos, I’m sure I’ve missed a number of important happenings from the year 2009.  If I have, please make sure to comment and I’ll add it if I have overlooked it. 

What does it all mean?  That is entirely up to you.  The year has brought many advances to augmented reality and the ceiling is sky-high for 2010.  There will be surprises, excuses, apologies, releases, re-releases and more.  Money will be pumped in, not all where it should go, and progress will be made.  We’ll laugh at the videos, smile at our favorite products and complain when it does work as expected.  Either way, it will be a year more interesting than the last and I’ll be here to watch it all unfold.

Ori Inbar (@comogard) tweeted an offer to join the discussion on the Outernet Guidelines Initative.  One particular point of their discussion piqued my interest and that’s the subject of privacy issues within the AR web (Outernet, ARNet, Digital Sea, etc., etc., ad infinity.)

John then hosted three panels, one at his office and two at the Web2Open (part of Web2.0) in New York City.  The focus was on how Augmented Reality would intersect with the Semantic Web.  John and Jack talked a lot about OGI (the Outernet Guidelines Initiative) and how things like AR would affect privacy issues (when image recognition comes into full fruition, is it okay for a stranger to map your face and locate your personal data?) business, (who has the air rights to the Outernet?) and how we think (if when I see you I can track your picture and know your details in the moment, do I even need to remember your name?)

The questions asked are important ones and they are absolutely right that we should be addressing them now before the potential use-cases of an Outernet become a legal squabble.  Who owns data is a tricky subject.  It will become more confusing when cameras and pachube items are harvesting information without anyone to decide if its acceptable. 

I believe we should have the following three rules about information harvesting:

1) Generalized information not tied to an individual is free to use.

2) Use of information tied to an individual must be opt-in.

3) Access to one’s own personal information is free. 

A simple rule-set to guide the use of information would help the creation of the Outernet.  Below are some use-cases of following the three rules of information and not following them. 

Ten Use-Cases for Following the 3 Rules

1) Rapid way to find lost children or lost dogs.  (no lost cats, when they leave it’s on purpose)

2) Worried parents can find out exactly where their teenagers went last night.

3) Teen-age girls can find out exactly what fashions are hawt.

4) Instant price comparision when shopping and would tell you where to find and if it was worth the gas to drive there.

5) Check if your favorite restaurant is busy. 

6) Business owners could figure out what everyone wanted and could carry less inventory which means less waste for society. 

7) Business owners could learn if people were happy with their experience in their stores. 

8 ) Shoppers know which stores have their particular item in stock. 

9) Drivers know the route to work with the least amount of traffic.

10) Grocery lists could be converted into maps to show you the fastest route through the grocery store.

Ten Use-Cases of Not Following the 3 Rules

1) Lift personal information and passwords when using kiosks (ATMs and credit cards).

2) Corporations can track individual tastes and spending habits to maximize profit on you, because they would know what you were willing to pay.

3) Government used data to decide if you might be about to perform a crime based on your facial expression (i.e. Minority Report or the TV series Lie To Me).

4) Estranged parents using the information trail to find the best place to snatch their children.

5) Churches checking up on their members to make sure they are not “sinning”.

6) Employers checking up on their employees to make sure they aren’t violating contracts.

7) Used to cheat in card games because “tells” could be identified by programs.

8 ) Stalk someone, and send them messages saying “I know you were <insert location>”. 

9) Health insurance keeping tabs on people for bad habits (smoking, eating fast food, etc) and increasing rates when found doing too many “bad” things.

10) Employers checking facial expressions for happiness quotient during customer interactions.  (ie – smiling)

Reagan famously said, “Mr. Gorbechev, tear down this wall!”

While arguments can be made whether Reagan’s challenge had any affect on the end of the Soviet Union, it still marked an important time in history.  In that same speech, Reagan referenced some crudely spray painted words on the wall–”The wall will fall.  Beliefs become reality.”  These were probably the more important words that he said on that fateful day. 

These words hold true for the current problem facing augmented reality smartphone app developers.  Beliefs have become reality for the VCs that any important AR apps must be made for the iPhone.  As Robert Rice, CEO of Neogence Enterprises, paraphrased what he’s heard, “We are only interested in deals that are… running on the iPhone….” 

The problem with this is that Apple isn’t giving up access to the video stream from the camera.  Current AR apps are either using the non-visual sensors like the GPS and accelerometer, or using single frames as a reference which make apps laggy and unreliable.  Once again, Robert Rice opines about this problem in his post–The Candy Apple Conundrum:

But I am continually perplexed by Apple’s refusal to open up the undocumented APIs in their SDK to allow augmented reality researchers and developers access to the video stream from the camera. This is a critical element for any type of visual tracking (markers, feature tracking, and markerless tracking). Some people figured some work arounds to make it work with the 2.x version of the SDK which is nice, except that this automatically disqualifies any applications from distribution in the app store. What really kills me though, is that on one hand, Apple made a play for welcoming AR applications with the 3.x version of the SDK, but only provided access to the overlay, while at the same time completely changing where the undocumented API calls were located, further obfuscating things and handicapping all R&D.

It’s clear that there is a high level of frustration within developers and Robert’s not the only one to notice.  Kim-Mai Cutler from Venture Beat talks about how, “[apps are] still being held back because they don’t have the right application programming interfaces for manipulating live video.”  And back in July, Ori Inbar posted an Open Letter to Apple. 

Why isn’t Apple opening up the video stream to developers?  Is it for the Machiavellian reason that they are developing their own secret AR master take-over as hinted by their patent filings?  Does any of this matter?

If we look at the Q2 2009 smartphone sales by Canalys, we see that the iPhone only holds 13.7% of the market.  While Symbian dominates with a whopping 50.3%.  What gives? 

The iPhone has done a wonderful job of controlling the branding space of future smartphones, but the reality is that they are only number three.  In this case, beliefs have become reality.   While I’ll be the first person to crow about augmented reality on the iPhone if they do ever release the video stream, I also don’t think we should be holding our breath any longer.  I find it sadly ironic that their iconic TV commercial from 1984 can be applied to them now.

Mr. Jobs, I am a small press blogger, but I sense the slow churn of animosity towards the iPhone.  The apps are what make the iPhone sell, and if AR takes off while you’re still sitting behind your wall, hype will drive them to a Symbian or Android smartphone that does support augmented reality. 

So tear down that wall, Mr. Jobs.  Or we’ll find someplace else to go that doesn’t have any walls.

The biggest barrier to true augmented vision is an HMD that can handle all the requirements.  I had the oppertunity to experience what Microvision, Vuzix and ORALab/EvoOpticks had to offer.  I’ll try to relate their talks and my experiences with their products and let you know if any of them have achieved augmented vision. 

Microvision – Ben Averch

Ben started the presentation during the Industrial section of Workshop Monday.  The first AR-style model Microvision tried was the Nomad which tried to be a hands-free automotive maintenance model.  The unit was plagued by poor ergonomics and eye strain.  It only used the red spectrum and they retired the product line in 2006 only two years after release. 

Microvision then changed their plans to achieve an AR enabled HMD by leveraging their military contracts.  The plan is to sell their Urban Warfare model (Ultra-vis) to the government and then use what they’ve learned to create a consumer model. 

They see a big market for a consumer HMD display but need to solve the remaining issues before its ready for the public.  I had an opportunity to try out their prototype HMD.  It wasn’t a full unit, but a bulky monocle on a stick.  While it didn’t give me an idea how it would function as a pair of sunglasses, it did give me a feel for the graphics.  I was impressed by the clarity of the color in the monocle.  The big problem I see for the first generation of HMDs will be the field of view.  This prototype only had a ~25degree field of vision.  The limited view will mean our early HMDs will only mimic our computer screens and not release the full spectrum of hands-free AR. 

Ben has written about his experience on his official Microvision blog and has said they expect a commercial product by 2011. 

It’s clear from the discussions I had with various industry members that a see-through wearable display that meets people’s expectations from both ergonomics and display performance is the big, obvious missing link in the AR solution story. We at Microvision are working hard to fill in this gap and create a technology solution that can allow this new market to take shape.

Ben also showed off their projector which at the time didn’t excite me much from an AR point of view.  However, after the key note speech from Mark Mine from Disnery Imagineering, I learned that the projector had a lot more going for it than I realized. 

Vuzix – Paul Travers

Paul gave us a nice history of HMDs including the failed hype of VR which has made getting money for HMDs harder to come by.  On a good note, Vuzix has a good track record of selling video glasses which gives them a good base to work from once they master the AR HMD.  Currently they’ve sold around 200,000 video glasses by marketing them the same way a HD TV would be sold.  This might mean that augmented reality’s best bet for widespread use is to be a Trojan horse within another accepted technology (similar to how Yelp gained AR eyeballs through their Monocle easter egg.) 

Unfortunately, Paul had to apologize a few times for misleading on the AR community about the “see-through AR HMDs” everyone thought were right around the corner with the 920Wrap.  He hopes they can reach market by 2010, but like Microvision they still have challenges to solve. 

I did get to try their video glasses, which were interesting, but I would have preferred to try out the glasses Ori tried at GDC.   Paul did say that see-through AR glasses would be priced from $199-349.  This sounds like a great price point that would help gain wider audiences.  I know I would buy one. 

ORALab/EvoOpticks

This presentation delved into more historical and technical realms rather than commercial usage.  Not a lot to pass along, but I did get to try their see-through HMD.  The view port was around 20 degrees and seen through two reflected mirrors on the screen rather than a laser created image.  They have no current plans for commercial production.   

And for the history buffs, the first patent for an HMD was by Albert B. Pratt in 1916. 

I wish Lumus had also attended ISMAR, but no such luck. 

Overall, I was disappointed that no HMD maker had a surprise announcement that they had a product ready for release Q1 2010.  Release dates of 2011 sound more like appeasement than real schedules.  Having experienced the products myself, I also realize the field of view will be a problem for early developers.  Don’t get me wrong, I’d be happy to have a HMD right now, even with a 15 degree field, but the small width will limit creativity.  Hopefully when they solve the lighting and weight issues they’ll also figure out how to achieve a 40 degree field of view. 

This past May, I predicted true Augmented Vision would occur around 2015.  After the demonstrations on Monday, I’m not going to change my prediction.  While I heard many quote Moore’s Law in regards to AR, I’m not buying it yet.  Moore’s Law is fed by billions (with B) of dollars which pushes the technology as fast as it can go.  AR is still languishing in VR’s dead space and has to catch a few wins of its own before it can start generating real cash to take advantage of Moore’s Law.   

Even if Microvision or Vuzix or Lumus release a commercial product in 2011, the software and apps to make the experience fulfilling won’t be available.  I also expect the first versions of the AR HMDs will only be bought by the early adopters.  It will take a few tries to make them lightweight, eye-strain free and with enough field of vision to keep people from having to rotate their head constantly.  I’m all for being wrong on this one, but I expect the road to widespread acceptance will be paved with a few missteps.  Cellphones took twenty years to go from the bag phone to the ultra-sleek smartphone, and while technology moves faster these days, I’m still sticking with my year 2015 prediction.