The HMDs are here!

The HMDs are here!

Okay, maybe it’s not that exciting, especially when you realize that the Mirage from Arcane Technologies is not a stylish pair of glasses or even some retro steampunk goggles outfitted with AR HMD gear.  The unit looks so bulky and bland that not even Lady Gaga could make it trendy.

But it IS an AR HMD.

But really it’s made for industrial usage and not everyday street wear.  Though I suppose, a truly hardcore AR enthusiast could augment their home and wear them safety inside without worrying about losing a chance to ever have another date.  Again.  Ever.

The system comes with authoring software so you can customize your markers.  I’d be curious if you could also use third party software like the Kinect to make it even groovier.  The Mirage + Kinect could become quite an indie hardcore hit.  However, since they don’t list the price on their website, I assume the cost is greater than even the most hardcore could afford.

Still, it IS another AR HMD on the market and to me we’re another step closer to stylish low-cost AR HMDs.  I’m still predicting 2015 as my arrival date for said glasses, but hopefully someone will surprise me and deliver a product sooner.

A few months ago Staffan Dryselius made a splash on Team Hack-a-Day with his DIY data glasses.  Since then he’s been working with a team to improve his design and would like to form a larger partnership with anyone interested in working on or owning a pair of AR glasses.  Having a working HMD for augmented reality would help the technology gain wider use.  Currently, we’re stuck with magic lens or web cam AR if we want to play with our favorite technology, though both have come a long way since early 2009.

The group is calling the glasses Maxcware (website not fully functional yet, but contact Staffan below if you want to join.)  If you’re not familiar with the reference in the name, I’ll give you a hint.  The name is from a science-fiction novel from this decade and if you haven’t read it, I highly recommend it (and it’s in the AR reading list.)  If you’d like to contribute to the group, contact him at staffan (at) maxcware (dot) com.

So to learn more about the project, I sat down to interview the man behind the vision (pun intended), Staffan, and since we’re talking about a visual medium here, I’m going to show you the glasses before we get to the interview.

Tom:

Interest in commercial HMDs has increased with spread of
smartphones.  Why did you decide to tackle this problem that the glasses
makers have failed to deliver on?

Staffan:

I had more or less despaired about any non-heinous, high-resolution
see-through HMDs emerging in my lifetime when Vuzix showed off their
new Wraps at CES 2009. When all they finally delivered to the market
turned out to be an opaque lump of plastic, I had finally had it. I
started to suspect that the public would continue to be spoonfed
incremental yesterware more or less forever. No single maker would
have the guts to make their inventory unsellable by launching anything
really nice and new unless forced at gunpoint. I would never get the
glasses I wanted unless I made them myself.

I first got excited about HMDs some time around 1995. There was lots
of interesting research done at the time, and also quite a few
companies advertising products “soon to appear in a store near you”. I
think it was around -96 something that Sony actually launched their
Glasstron model, and there was also the “Olympus Eyetrek” soon
afterwards. I however decided to wait a bit, great things seemed to be
just around the corner. Especially one company, Digilens had an
awesome idea for optical see-through AR-type displays using switchable
Braggs gratings in 98-99… I was very excited at that one especially
(the company has by the way resurfaced as SBG Labs with yet another
vaporware design).

Then the dot.bomb exploded, and everything digital died. “Virtual
Reality” became “Definitive Nonexistence”. The headsets by Sony and
Olympus were phased out, and the Digilens homepage died shortly after
they decided to do fibernet switching chips rather than HMDs… That
was more or less the situation for many years, and I was very
disappointed and soon promised myself to try and forget all about
HMD:s until I saw an advertisement for something really good I could
actually buy in a shop.

Since then, I have read several science fiction books featuring HMDs,
seen the developments in smartphone AR emerge and again felt
frustrated about the non-existence of  useful HMDs.

Enter 2009 and CES. Vuzix were showing their new Wraps. Wow! At last!
I couldn’t wait for the release date for their fabulous new optical
see-through displays! The disappointment was what made me do it. Even
though I didn’t really know how, I had to give it a shot.

Tom:

From the picture, the screens appear to be non see-through.  Is there any
possibility of making them see-through so true augmented reality can be
accomplished?

Staffan:

Yes. And that is the plan too, of course. The first step is to add a
camera to the glasses to feed the display ambient video blended with
digital content. It is much neater not having to point the camera of
the phone itself around to use AR applications in the glasses, as must
be done today. As soon as possible we will also want to add
accelerometers and magnetometers to the glasses.

The top half of the glasses will continue to be completely clear.
There is no need to expand the physical screens any further, only the
virtual screen estate. Those two are quite separate entities, but it
is only when keeping the optics sufficiently close to the eyes that
this becomes really obvious. It’s like peeping through a keyhole: Keep
your eye close enough and the aperture lets you see the whole room.

From the beginning I saw the “see-around” (or rather “see above”)
design combined with “really near eye” optics as just a pragmatic way
to make something useful with available technology. However, a very
nice aspect of the “really near eye”-design is that the physical
movements of the eyes can actually become useful instead of being just
another engineering obstacle. It is especially useful that the eyelids
work as natural shutters, switching to the view that is most
appropriate for the moment. When looking straight ahead or upwards,
the lower eyelids completely block the screen so that light from the
displays doesn’t disturb the natural vision. When looking down, the
upper eyelids block lots of the ambient light that may otherwise
bleach the screen.

Apart from a convenient way to keep alive when traversing a street, it
also means that camera see-through becomes practical. At first, the
mere thought of camera see-through made me shudder. Although that
solution can more or less immediately be used together with Layar and
all the rest of the applications for smartphones, both limited field
of view and latency are fierce problems to combat without a
possibility to momentarily switch to complete see-through. The latency
may not seem too bad at first, but try and navigate while walking at
any speed using only the viewfinder of a video camera. Fixing a camera
to the glasses is far worse and reacts to every jerk of the head. To
keep the screen from bobbing about, you have to take it real slow… If
motion sickness is not enough to make a person reconsider, then the
inevitable robotic choreography should inspire second thoughts about
testing the concept in public.

Tom:
How do you envision the use of these glasses?  Hooked up to an iPhone or
Android (or whatever smartphone you use) to project the screen realtime?  Or
some other usage?

Staffan:

All that is needed is connectivity and some basic sensors. They’re all
there in today’s smartphones, so yes, the glasses will hook up to
them. As many different makes as possible and as easily as possible.
As for uses… Wow! Where to start?

…Humanity is a little like the first amphibians. We’re popping our
heads above the surface of the primordial soup right now. There is a
completely new digital world in the making out there. We are just not
very well adapted to take part in it, and the interfaces we use today
are laughably inadequate for interaction. With AR glasses we may at
least get up from our asses and shut the door on the cubicle. Reading
company spreadsheets can be done just as well on the way to the beach.
The best ideas may come to our mind when we are in the supermarket,
only today we forget before we’re back at the computer. No more so. A
digital post-it or email is quickly edited in the corner of the eye.

But work and “productivity” is boring… Instead Google should be there
with us when we see a new butterfly in the park. Getting lost in the
city in the age of GPS? –That’s laughable! Directions should be where
they belong, as AR overlay. No more getting scammed in a shop. The
barcodes should trigger balloons with user tests and best prices on
the go. Blogger? -Updating the skateblog should be done when we are
actually up and rolling, complete with action footage and biometrics.
Why make do with just the normal senses? Nightvision? –No problems.
X-ray vision may come in handy while sharking by the pool, just pop
out the IR-filter if you are so inclined. Bad-hair-day? –Put on a
digital wig and a happy face. Bored? –Just connect to a robocam in a
Tokyo bar. Going to a meatspace party? Bring your avatar buddy along…

Starting to sound outlandish yet? SciFi? I say all this is very close
at hand, and we just need to light the match to set the digitality
ablaze. It is long overdue…

Tom:
You mention on the hackaday post that the image is doubled on the two
screens.  Have you figured out how to split the image to get the true 1280
width?

Staffan:

More or less. We will probably want to device a completely new
graphics card instead of doing too many hacks on the original MyVue
PCB, but it is doable even on that one. I’m just afraid that we may be
wasting valuable time going down that alley too far. I think it will
be better to put something together that doesn’t require un-human
soldering skills to work. I want to put together a more manageable kit
instead so that as many people as possible can get involved. The Kopin
displays are however well documented, and there is no magic involved
in interfacing with them. I  have a friend working on it, but don’t
want to push it. In the Hackerspace groups I also mention, we are
getting better organized. There is now a webspace up and running (for
our internal purposes as of yet), and we are putting together a
“to-do”-list allocating work-packages for the different members. The
front-page of Hackaday gave some new contacts too, and I’m having
serious pangs from my conscience for not handling them yet! I’ve been
lazing away with the family doing things like sailing and the like…

Tom:

How much would it cost if someone wanted to make their own pair?

Staffan:

A pair of MyVu glasses cost about $150 on eBay. Add some Fimo putty
and a pair of oversize-sunglasses (the kind that fits over regular
glasses) plus a couple of days work (depending on skill), and you have
a crude but passable pair.

If you want something better, you may download the meshes for the pair
I have and order better frames from an online prototype maker. I don’t
have the figures for how much that would be, but that is a quite
expensive alternative. Better then to wait until I can fill an order
with a Chinese factory. A box with a 100 pairs will cost about $100 a
pair.

There is then the video card, better battery and case… No figures there yet.

Tom:

Why are you going about this as an open source project?

Staffan:

Further, those who do understand say I’m either mad to disclose
everything on the net and to loose an excellent business opportunity,
or call me names for destroying the patentability for others. I
usually retort that the industry hasn’t moved at all for a decade, and
why do they believe I would fare any better? I also try to line out
the difficulties with classical innovation processes. I’ve been there,
on both sides of the fence. As inventor and as executive in a joint
industry-governmental innovation system. I know how bad it can be.So
much time and creative drive can be lost in anger over incompetence,
greed and dishonesty that you simply don’t want to think at all about
your project.

I believe that instead of getting entangled in patenting processes, VC
negotiations, hunting for (competent!) technical consultants,
marketing and manufacturing partners, it would be far better to copy
some applicable concepts from open source software development. With
the glasses, I want to perform an experiment. I would so much want to
put together a really nerdy team of developers that are driven by the
fun of problem solving and a feeling of contributing to a community
rather than for direct economic benefit. Not that there may not be a
chance to make some cash one day for everyone involved, only the money
should not be the driving force.

Here is a good clip to illustrate what I mean:
http://www.youtube.com/watch?v=u6XAPnuFjJc

Naturally, hardware is much more difficult to develop in a community
than software. However, I believe the time may be ripe for a test of
the concept. If any project can succeed, I think AR glasses is it. The
time should definitely be ripe for technologically inclined people to
want those for themselves. I know I definitely do. The basic concept
is also in place, and both PCB:s and plastic/mechanical components are
quite cheap to manufacture these days, even in singular quantities.
Many of the potential combined early adopters and developers will be
able to make their own glasses and feed the loop.

I can see an emerging ecosystem where different participants can
specialize and even start to make some money from selling
non-complicated sub-systems. There is also potential for spin-off
projects and services that will benefit from AR-glasses. There are
many angles to this experiment…

** End of Interview **

Whew.  I agree, Staffan, the time is ripe for a technologically savvy group to tackle the AR glasses problem.  And given the importance of this little piece of hardware to the overall AR ecosystem, I think it’s worth our time and hopefully worth your time to join this project if you have something to give in the way of knowledge, expertise or time.

So stop by Maxcware or contact him at staffan (at) maxcware (dot) com if you’d like to contribute.  Or at the very least, sound off your encouragement at Games Alfresco.

The year 2009 has been an important milestone in the development of augmented reality as an important future technology.  While markers and marketing campaigns captured the early interest of the year, the bigger story was the porting of augmented reality to the various smartphones in the second half. 

But those two generalities aren’t the only thing that happened in 2009 in regards to augmented reality.  I’m going to take a look back at the year, month-by-month, to remember how the year unfolded, so we better understand the direction of the future. 

January

Back at the dawn of 2009, all the augmented reality news you needed to know was in one place – Games Alfresco.  Hope was high and all the pieces seemed to be clicking into place. 

Biggest News of the Month – Metaio releases its Lego AR boxes

   * By a long shot, Metaio’s release of the augmented reality Lego box was the biggest news of the month.  When you can get Angela Merkel and the Govenator to pose with your product, you’re doing well.

Coolest App/Video – iPhone App Helps Solve Rubix Cube

   * This app portents of things to come in the future as products like Google Goggles amp up the possibilities of reality-search.  Using specific algorithms (agents) for specific tasks, there will be an app for everything you want to do in the future.

Best Article / Interview – UgoTrade Interview with Robert Rice

   * Though Robert hasn’t delivered on his promise of a Neogence product by mid-year (though the Apple video API has stymied their progress), the rest of the interview is full of great quotes and nuggets about augmented reality.

   * Notable quotes from Robert that show how much he understands the industry:

But yes, I’d say that the next 18 months are going to be very interesting with a lot of money being thrown around, new ventures, and plenty of content/applications. I expect most of this will be centered on single user AR experienced through a mobile device with a screen (iPhone, android, etc.). I expect that there will be a significant boost after Vuzix releases some of their wearable *transparent* displays, putting Microvision back into the “has potential but is too quiet” position.

Remember, one of the biggest risks that AR has, is being branded as “novelty”, which means “cool for five minutes but ultimately a waste of time.” I think we have a ways to go before something is truly useful, but as 2009 progresses we should start seeing some effort here. I’d guess 2010 before something really useful comes out…at least something practical.

Now, having said that, I should say that I expect entertainment and games to take the lead (as usual), although there are a few companies really trying to leverage AR and video/graphics compositing for marketing (brochures) and location based methods (kiosks, large screen projections, etc.)

Other notable things that happened in January – The Augmented Times is born and Rouli Nir begins a great year of chronicling the happenings of AR. 

An auspicious quote from Rouli:

This blog is about Augmented Reality (AR). It is my firm belief that AR will be the next web revolution (the so called Web 3.0, and forget about that semantic web nonsense). In the next few years, stronger devices and better algorithms will enable us to merge the real world with cyberspace. Using your mobile phone (at first) or head up display (later on) every real object will be augmented and achieve a web presence. Tourism, shopping, advertisement, entertainment and education are only a few areas that will never be the same.

This blog mission is to document this revolution as it happens. We welcome you, and welcome our augmented future.

February

This month is relatively quiet due to it being handicapped by fewer days than the other months. 

Biggest News of the Month – Augmented Reality in Flash Now Commercially Available.

   *  Making the nuts-and-bolts tools of AR available for commercial use is always a good thing, especially when its paired with flash. 

Coolest App / Video – “Sixth Sense” MIT Device.

   * Patty Maes unleashed this phenomenon on the web early last year and it still gets links and mentions today.  While projection based AR is probably not medium that we’ll all eventually use, the video showed tangible use-cases that help the non-initiated understand the embryonic technology.

Best Article / Interview – This comic from the Abstruse Goose sums up a lot about the future of ubiquitous computing (via Augmented Blog.)

Other notable things that happened in February- I started the Future Digital Life blog, which upon reading my early posts, makes me cringe a little.  Oh well, nothing ventured, nothing gained.  Also, Rouli starts the Weekly Linkfest, the best place to find the round up of AR news each week.

March

In like a lion, out like a lamb.  Augmented reality starts to get rolling in March with lots of goodies and promises.

Biggest News of the Month – Metaio and Vuzix Team Up to Create AR Goggles.

   * While the year has been mostly disappointing regarding HMDs, this was the one tangible piece of progress in the bunch.  The video Ori captured at the GDC portends to greater things.  While no companies have released products for the Vuzix+ARCam, we’ve recently seen an impressive project from Craig Kapp with his Whisper Deck. 

Coolest App / Video – Blair MacIntryre’s GA Tech team – Zombie Attack and Pit Strategy.

   * The videos showed what polished AR games can look like. 

Best Article / Interview – I’m going for a two-fer on this one because its hard to choose between Ori’s talk at WARM09 and Robert’s Decade of Ubiquity post. 

   * If you haven’t watched Ori’s talk, I highly recommend it.  Even though I’d seen the video, I made a point of hearing him live at ISMAR09 because I think its such a well thought-out call for human-based technology.  Robert’s vision is a bit more expansive and covers the whole of augmented reality.  I’ll leave you with another Robert quote:

We will break away from the desk, we will throw away our monitors, and our children will laugh at how large our IPhones are. They will struggle with how we ever managed to get work done with “windows” “webpages” and keyboards. They will be unable to fathom the concept of vinyl disks, typewriters, and landlines. But it all starts, and accelerates, during this next decade. Imagine everything that happened in the last decade, and multiply it. You haven’t seen anything yet. The next decade will make the last one pale in comparison.

Other notable things that happened in March – Topps released augmented baseball cards and ISMAR09 begins calling for papers. 

April

Biggest News of the Month- Rouli and Ori team up on Games Alfresco to create the AR hub for all AR related news.   

   * The move allowed Ori to focus more on his company and gave Rouli the helm on the daily AR newscasting.  Looking back at the great job that GA has done, I’d say it was a good move.

Coolest App / Video – Marco Tempest’s AR card trick video.

   * The video hit the web in April and showed us how AR can change our perception of reality. 

Best Article / Interview – Tim O’Reilly talks Web 2.0

   * Augmented Times identified the best quote from the video:

RFID is an evolutionary dead-end … semantic web or RFID is things “wearing name-tags”, and web 2.0 is learning to recognize things … We’re getting to that kind of augmented reality, where our computers will have senses that are as good as ours or better … they are going to recognize faces, they are going to recognize objects, they gonna have immediate recall. If you ask me “what’s the UI in five years”, it’s a pair of glasses … I’m gonna have some kind of little heads up display because I’m gonna look at something, I’m gonna walk around at a meeting and it will go “that’s Joe, you met him three years ago”.

Other notable things that happened in April – Georgia Tech fear of heights video (another peak into the future of augmented reality as a perception changer), Rouli’s call for marketers to stop using AR to sell cars, Lester at the Augmented Planet gets his blog started, Nokia point and find is out and a couple of articles about brain-computer interfaces: Honda controls robot and the brain-twitter breakthrough. 

May

Biggest News of the Month – Metaio releases Unifeye design demo.

Coolest App / Video – Useful AR from the US Postal Service.

   * Using FLARToolkit, the US postal service shows us that not all AR applications have to be novelity ones. 

Best Article / Interview – Tish interviews Ori Inbar about mobile augmented reality. 

Ori: Just like with every emerging technology in history, people never bought the technology, they bought the content, the apps, the benefits that came on top of the technology. Whether it was VHS winning over Beta Max, or BluRay winning over HD. It’s always because of more/better content. Look at the video game console war: Xbox, and Nintendo did better than Sony just because they had more and better games. Even Windows was a success thanks to its applications. People bought it for the applications not the OS. The content is the first to drive demand.

Other notable things that happened in May – My post about the Path to Augmented Vision, Programmer Joe talks AR at LOGIN, and 11 Industries to be Reinvented with Augmented Reality. 

June

Biggest News of the Month – Layar is online. 

   * The biggest news of the month by far.  Layar was the first big product release that got a buzz bump. 

Coolest App / Video – Zugara Social Shopper.

   * The video showcased the use of motion capture to help eliminate the burdensome need for keyboards and mouses when you’re using augmented reality.  Zugara later put the technique into practice for their Cannonballz and Fashionista products.

Best Article / Interview – UgoTrade Interview with Blair MacIntyre

   * No surprise here.  Tish Shute continues to get great interviews with everyone important in the AR field. 

The problem with the mobile phone as a AR device is that problem of awareness. If I have a head mount on and I walk down the street and there is bunch of probably-not-useful-but-potentially-useful information floating by me, that’s a good thing, because I may see something that is useful or makes me think of something else.  But if I have to hold up my phone to see if something might be interesting nearby, I will never hold up my phone because at the time there is a high probability that there won’t be anything particularly important there.  You might imagine you can get around this by using alerts or something like that, but then you overload whatever alert channel you use.

Other notable things that happened in June- Ori asks us all about our favorite AR games of all time, Noah breaks onto the scene with his Touchless Glove Interface, and Goggle presents their paper on Computer Vision and Pattern Recognition (which later becomes Goggles). 

July

Biggest News of the Month – The AR Consortium Announced. 

   * While its still unknown if the consortium has accomplished anything through cooperation, the intention of cooperation is a start.  Here’s to hoping we’ll see more fruit from this vine in 2010.

Coolest App / Video – AcrossAir Tube Finder.

   * At the time it was released we weren’t sure if it was a real product, but eventually we learned the truth and for a time, AcrossAir became the top selling AR app on the iPhone appstore. 

Best Article / Interview – UgoTrade Interview with Ori Inbar “Reality Reinvented”

Ori:You are so right. It’s multiple elements in the industry that have to come together. You have the technology companies like Imagination and Metaio, then you have content companies like what we’re trying to do; the hardware vendors and the large content providers. Those brands that we’re trying to go after and educate them about the potential of AR. All these pieces need to come together for this market to ignite.

Other notable things that happened in July – James Alliban’s business card goes viral and AR Avatar toys are released at Comic Con.

August

Biggest News of the Month – Yelp Introduces AR Monocle via Easter Egg

Coolest App / Video – Bruce Sterling’s Layar Keynote – At the Dawn of the AR Industry

   * Bruce released an AR-infused novel Catyadids in 2009, so it was only natural that Layar asked him to speak about the fledgling industry.  The speech is memorable with too many great lines to pick from for quoting here. 

Best Article / Interview – Thomas Wrobel’s Proposal for an Open AR Network

   * Easily Tish’s best interview because it turned into more than a reflection of past events or hopes for the future.  Tish and Thomas have since rallied AR professionals and amateurs to join the AR Wave collaborative effort.  They seek to use Wave as the platform for an open AR network and have recently been testing code. 

Other notable things that happened in August – AR Strippers!

September

Biggest News of the Month – Vuzix 920Wrap Will Not be See-Through

   * The biggest disappointment of the year was the news from Vuzix.  Good see-through AR glasses are supposed to release us from the tyranny of the “magic lens”.  Hoping 2010 brings better news, but I’m really waiting until 2011. 

Coolest App / Video – Augmenting Aerial Maps with Dynamic Information

Best Article / Interview – Gene Becker’s Design Strategies for Magic Lens.

Other notable things that happened in September – No more Gamaray - Death of a Browser, Int13 and Total Immersion getting into bed, and the augmented reality reading list.

October

Biggest News of the Month – News from ISMAR09, News from ISMAR09 #2 and UgoTrade ISMAR09 Coverage.

   * I was lucky enough to attend and speak at ISMAR09.   The energy of the participants was exceptional and I came away from it more jazzed about AR than I was going in.  Hopefully the ISMAR committee will be releasing the videos from the event soon. 

Coolest App / Video -  2D sketches become 3D reality. 

Best Article / Interview – ISMAR09 HMD Review

   * I think I summed up the current state of HMDs for AR pretty well.  I’m hoping 2010 brings a suprise for AR HMDs.

Other notable things that happened in October – AR Wave discussion on UgoTrade, Wikitude for the iPhone first look and Giant Hand Torments City

November

Biggest News of the Month – Junaio hits the app store. 

Coolest App / Video – Interactive Entertainment Using AR

   * The videos about the Disney experiences were similar to what Mark Mine showed us at ISMAR.  They really show us how technology can transcend our perceptions of reality.

Best Article / Interview –  Augmented Planet’s Head-to-head browser test (and part 2)

   * Lester broke down the various browsers in this excellent hands-on challenge.  Read both parts to find out the winner.

Other notable things that happened in November – Business Week’s Article about: Getting Beyond the Hype and Robert Rice’s reply. 

December

Biggest News of the Month – Goggle Googles is released.

   * The giant has awoken.  Goggles has many implications for Googles ideas on the AR market.  This won’t be the last time we’ll hear from them about this technology.

Coolest App / Video – Layer 3.0. 

   * While Layar has since pulled the app from the app store and had to issue and apology, the version 3.0 still holds lots of promise. 

Best Article / Interview – Augmented Reality in 2010: Predictions (10 part series)

   * A wide swath of the AR blogosphere had the opportunity to participate in the predictions series from Games Alfresco run by Rouli Nir.  More interesting, snarky, and possibly-true predictions that you can shake an augmented stick at.

Other notable things that happened in December – AR Wave FAQ on UgoTrade, Enkin acquired by Google, ARDevCamp, Wikitude Teams Up with Lonely Planet, 10 Worse Uses of AR in 2009, and Nexus One News and Implications of AR. 

What It All Means

Whew.  After reviewing hundreds of articles and videos, I’m sure I’ve missed a number of important happenings from the year 2009.  If I have, please make sure to comment and I’ll add it if I have overlooked it. 

What does it all mean?  That is entirely up to you.  The year has brought many advances to augmented reality and the ceiling is sky-high for 2010.  There will be surprises, excuses, apologies, releases, re-releases and more.  Money will be pumped in, not all where it should go, and progress will be made.  We’ll laugh at the videos, smile at our favorite products and complain when it does work as expected.  Either way, it will be a year more interesting than the last and I’ll be here to watch it all unfold.

Head-mounted displays are the killer hardware that augmented reality needs to break free from “magic lens” smartphone technology.  SAP TV shows us an industrial application concept at a Dailmer AG plant.  The part stocker is using an HMD with augmented reality to keep track of the correct parts (their wording makes it sound like this is a concept only.)

The concept screen the worker sees looks like:

If this technology can be realized then plant floors can make big improvements in efficiency and cost by cutting down on expensive errors.

The SiWear research project is sponsored by the German Federal Ministry of Economics and Technology. Partners are SAP Research, Daimler AG, teXXmo GmbH; The Mobile Research Center and Neo Business Partners GmbH.

ISMAR09 Sympsium & Expo is right around the corner and I’m pleased to announce that I’ll be not only attending ISMAR09, but I will be presenting during the Manufacturing section on Monday and joining the panel discussion as a representative of Toyota.  I’ll be reviewing the possibilities of augmented reality in an industrial setting, including a project that I’m working on with Metaio.  Hopefully I can enlighten participants on the constraints and challenges of implementing augmented reality on the factory floor. 

I’m announcing this on my blog, so if anyone has any requests for topics during the discussion, I can be sure to bring them up.  I was only able to confirm attendance last week, so there’s still plenty of time to update my presentation. 

Not only am I excited about presenting, but I’m more excited about getting to meet a lot of the people I’ve interacted with throughout the AR world.  I’ll be there from Sunday until Tuesday afternoon, so if you’re going to be there then, I’d love to meet up, chat and maybe, share a drink.

To reach me before or to contact me during the conferance, email me at tom.carpenter (@) tema.toyota.com or twitter to thomaskcarpente.  (Yes, the name is missing the ‘r’ since my name was too big for twitter.)  I’ll be tweeting updates during the conferance (as well, I use it regularly for posts and other augmented reality related news.)

This past month the specter of augmented reality, which in the past had been mostly research papers and speculation, has slowly been turning into real products.  With a nice long three day weekend to relax, I thought I’d reflect on the more important happenings of augmented reality.

Augmented Reality Apps

Yelp

The hottest topic for augmented reality apps was the easter egg in the iPhone app Yelp.  With three shakes of the iPhone, the Monocle application would be unlocked allowing camera based AR.  The impact of this application on the news can be seen in this technorati tag tracker:

Wikitude Drive

Wikitude, the creators of the first reality browser, gave a preview of their augmented reality navigation system for the Android.  The navigational system called Wikitude Drive overlays point-to-point directions on the camera view.  Some concerns were expressed about safety for the driver, but this application is no different than any navigation system already on the market.  And for you iPhone junkies, don’t worry, they’ll be releasing it on that smartphone as well. 

And while I’m on Wikitude, they also released Wikitude 3.0, an enhanced version of wikitude.me.  Read the article for more details. 

Layar 2.0

Competing for the mindshare of future “reality browsers”, Layar has done a good job of getting its product noticed even though it came six months after the Wikitude app.  Layar is already onto version 2.1, making improvements to its browser like linking within Layars and sharing screenshots. 

Put-A-Spell

Living up to his promise of AR education games for kids, Ori’s company Ogmento brings us a demo of their Put-A-Spell game for young children. 

iPhone OS 3.1

Of course, this operating system isn’t an app, but its precursor to an assumed flood of AR apps we’re expecting to see when 3.1 finally becomes live.  Expected in September, grumblings of delays have been speculated based on progress in beta.  When it finally does drop, expect to see a huge spike in augmented reality articles. 

AR Goggles

Lumus

While we’re still a ways from augmented vision, its still nice to track progress of augmented reality goggles.  Ori Inbar gives us a peak into the progress of AR goggles in an interview with Zvi Lapidot, CEO of Lumus.  While we can’t expect to see a commercial product from Lumus until 2011, mostly due to a lack of VC funding, we can take a look at this demo from them.

Vuzix

While we’re still waiting for the AV920 Wrap to be released this fall, Vuzix has updated their website with a page with an AR Education Group.  This group will focus on training services for various industries from medical to military with both off-the-shelf training and custom. 

Nokia

At the recent SDForum, research fellow Kari Pulli, talked about Nokia’s take on mobile augmented reality.  They believe the future to be, not on handsets, but in AR goggles and are developing a prototype model.  The slide show can be seen here.

Interesting Articles

Eight months ago, I could keep up with all the discussion easily.  Now many new hats are being thrown into the ring.  I’d like to point out my favorite articles from around the ARNet.

Games Alfresco, Ori Inbar – Why People Get Excited About Augmented Reality  

Ori gives us a nice rundown of the influence of science fiction on technology and how augmented reality will bring us: x-ray vision, time travel and teleportation.

Ugo Trade, Tish Shute – Everything Everywhere: Thomas Wrobel’s Proposal for an Open Augmented Reality Network

Showing what a great discussion Thomas unleashed, the comment section was nearly as long as the article itself.  As a frequent contribuitor on the AR blogs, it was nice to see Thomas outline his vision for the future of augmented reality. 

Bruce Sterling – At the Dawn of the Augmented Reality Industry

This video was recorded at the Layar Launch Event.  Bruce Sterling, a promenant sci-fi author who helped bring about the cyberpunk movement, gives us a rundown of down-market cheesy AR, who’s leading the AR development race, what to call augmented reality and other ramblings on a fledgling industry.  Bruce is also writting regularly about augmented reality and other wired topics on his blog Beyond the Beyond. 

Marshall Kirkpatrick, ReadWriteWeb – Augmented Reality: 5 Barriers to a Web That’s Everywhere

The challenges to an augmented reality world are more than the sum of its parts.  Marshall breaks down these challenges in five parts: Social vs. Real-Time, UX, Spam and Security, Interoperability and Openness. 

Rouli Nir, Augmented Times – Looking for a Modern Day Chaplin

Rouli explains why we need more storytelling and mystery in our AR applications.  As he says, “content not technology is the way forward.”

This hits a nice cross-section of my favorite articles from the last month.  I also would like to point to the three articles on my own site that got the most traffic: Proposal: Augmented Reality Scale, 10 Things Your AR App Must Have to Succeed, and 10 Franchises That Need Augmented Reality.

I’d like to say that I discussed all the major happenings from the past month, but the volume of AR information coming out lately makes that impossible.  For the items I have missed, I suggest checking out the Weekly Linkfest on Games Alfresco/Augmented Times (of course, most of my readers, also read that site, but for those that aren’t yet):

 Weekly Linkfest – Sept 6th

WeeklyLinkfest – August 30th

Weekly Linkfest – August 23rd

Weekly Linkfest – August 16th

Lastly, if you’re still not on the augmented reality bandwagon, I present to you the most recent augmented reality trends graph on Google trends.

For some reason this augmented reality video from a couple of years ago is making the rounds again on various blogs and even Gizmondo is reporting it as fresh news.  Since its resurfaced, as a fellow car engineer for Toyota, I’ve decided to address some of the challenges to making a system like this really work.

1. AR Glasses

The first and obvious challenge is that the glasses aren’t yet a reality though they will be this year or next.   While a mechanic doesn’t care about style, the glasses still have to function well.  How fast of a refresh rate will the glasses need to not create “head lag” as the mechanic looks around in the engine.  The controls for the unit will also matter, since they’ll need to stop, start and rewind. 

2. Orientation

Without a paper marker, the glasses will need to orient the graphics onto physical locations on the car.  How well will they respond to dirty motors, poor lighting or the mechanic getting in the way of the camera?

3. Spatial Issues

When doing maintenance on parts that are easily accessible, augmented reality works well, but what about when the water pump under the wheel well needs replacement?  How well will augmented reality project the proper maintenance steps deep into the car engine space? 

4. Filling the Database

In the early nineties, our plant purchased a “wonderful” maintenance system that would house all of our PMs, keep track of all parts and what machines they went to and make sure we kept our machines in tip-top shape.  The flaw in this wonderful idea was that someone had to actually put all this data into the system and maintain its integrity as we upgraded machines and moved equipment around.  For high-frequency work, these AR maintenance glasses would be easy to develop, but who is going to make the other two hundred infrequent displays needed?

Overall, the promise of using augmented reality in industrial situations is quite large.  Given an improvement in the technology, I could find dozens of applications in my own plant.  Visualizing difficult tasks for team members using 3D data has huge potential, but it has to work right and be cheap enough to make it to the plant floor.  While I think AR will make its way into the production of cars (in fact, I’m working with Metaio on one such application) many issues need to be solved to make it a widespread usage.

In all the hubbub about celebrities dying the last few weeks, I somehow missed that Pachube started a blog about their work.

Okay.  I really wasn’t distracted by the celebrities, but we do have a new source for AR information on Pachube’s blog. 

Mostly, we’ve been discussing the software side of Augmented Reality.  Most of the applications we’ve seen of AR have been in the realm of marketing with the occasional browser like Layar or Nokia’s “Point and Find.”  But all these types of AR require human interaction to input the data for it to be visible.  Pachube aims to be the connection of the real world to Internet through sensors to help create the Internet of Things. 

They recently released a video on their blog showing how real time data can be shown on AR displays.

Though I would have to convince our draconian IS department at Toyota to allow us the newer smartphones instead of our Blackberrys to make this work, I’d would find a lot of use for in the plant.  Often we’re forced to use expensive monitors on our casting machines to display data.  And if you want to change that data to a different format, forget it, because its extremely hard to change.  Using Pachube style interfaces with a smartphone, it’d be simple to walk by a machine or production line, point my camera at it (preferably using GPS+compass rather than markers) and get stats instantly. 

I also see Pachube type sensors allowing for lots of other interesting applications.  I had talked about some of them a few months back in a few posts called Machines That Know, Good Things and Bad Things.  Now I was mostly talking about using cameras with advanced object recognition, but sensors can work in similar applications. 

Overall, happy to see Pachube starting a blog and I’m looking forward to regular updates on their endevours.

The following video from Immersion R&D shows the manipulation of a 3D object using their Cubtile device.  I’m not particularly keen on the cube because it seems more like a prop out of a bad science fiction movie rather than a real interface device.  I really couldn’t imagine having one of those in my house or at work. 

However, the manipulation of the 3D object shows off nice usage of AR in a hologram sort of way.  The interesting part of the video comes at 1:08 when she pulls the object out of the cube. 

Using a haptic or air-touch system, 3D object manipulation could be useful for designers that need to see all views of an object.  But they will also need a way to select individual surfaces, lines or points to make real changes. 

Having watched many of my engineers or techs manipulate 3D objects, I can say they spend a lot of time rotating objects around to get the proper view.  Using AR glasses with a freeform control that feels natural (not the cube) in their resident 3D program (design, manufacturing, etc), they could improve their development time by a significant amount to make it worth the additional cost of the AR program. 

Unfortunately, this means unless a universal AR viewer can be applied to any 3D program, each product developer will have to incorporate AR into their product.  We use a wide variety of programs at our one facility: MasterCam, CamTool, Pro/E Wildfire, CADLite, etc., so the cost could skyrocket quickly, making any benefits cost prohibitive.