About a month ago, I proposed an augmented reality scale to help us define different applications of the technology.  While the post received good feedback, I felt that it could use more refinement. 

This update will define the scaling between 1 and 10 for the Reality Recognition axis.  This will help others use it, as previously, the definitions behind numbers on the axis were too fuzzy and therefore not intuitive enough to be usable.  I’ll give more definition to the other axis, Perceived Reality, in a later post. 

First, a refresher on how the scale works.  The RIM scale is composed of two axis: Perceived Reality (PR) and Reality Recognition (RR).  I chose two axis because AR exists through the mixing of reality and the virtual.

The Perceived Reality axis shows us how the graphics are indistinguishable from reality (on a scale from one to ten).  The Reality Recognition axis explains how completely computers understand the world (on a scale of one to ten).  A total score for an application of the augmented reality concept can be given by using the two numbers as a vector (#,#).

Previously, I only gave a few points on each axis giving a general idea of how to score a particular application of the technology.  Now I would like to refine that.  On the Reality Recognition axis, we’re trying to define how computers see the world.  I broke the axis down into the three types of objects the computer will need to understand:

1) Non-moving structural objects - buildings, trees, landmarks, roads, lightposts, signs, etc.  Anything that can be constructed and doesn’t move regularly.  These will be best defined by GPS systems since they do not move.  They will form the outline of the augmented world. 

2) Moving objects and people – Cars, cans, books, posters, desks, people, dogs, etc.  All the pieces between that move.  They will be best defined by local object/person recognition. 

3) Information - the invisible data that exists without an object.  Temperatures, pollution levels, CO2 levels, ocean currents, etc.  This is information that can be collected by a sensor (Pachube or other data collector), but not information that defines an object (information about an object will already be tied to it).

Each category (Non-moving, moving and information) has three degrees of implementation.  Simply put, each one can be low (1) , medium (2)  or high (3) implementation. 

Non-moving Structural Objects

A low implementation value for a structural object like a building would mean that only the general GPS location was defined.  One could find the building and attach wiki or other information within an AR network, but you could not know its shape and size.  A medium implementation value would give size and shape in a general manner.  A high implementation would show the building down to the finer details and might require local object recognition to define it. 

All current Nearest-X type of AR apps are at a low implementation for structural places like a Starbucks or a subway station.  As projects like Microsoft’s photosynth or Google Earth define the size and shape of the world, then this category will move to medium implementation.

Moving Objects and People

For moving objects, we’re still within the primitive low implementations because object/person recognition works only on narrowly defined applications.  Sein’s SREngine, Sony’s Vision Library or Zugara’s motion capture game Cannonballz use elements of local recognition.  A medium implementation, within a category, could choose between a large data set. 

For example, at medium, computers could recognize any face on the planet (given a data set to compare to).  But at medium levels, we’ll have to be concerned with personal freedoms because computers can then be used to track people much the same way usage habits are currently tracked on the Internet. 

Information

The last category Information is the least advanced of the three.  Mostly this is because the hardware side is both limited and expensive.  GPS data is supplied by the government and cell phones give us a handy tool to utilize it for structural places.  Object recognition requires a camera, but since cell phones have them and are ubiquitious, this data is also accessible. 

Information like pollution data or temperature sets requires a sensor to collect it.  While the world is filled with billions of sensors, most are not connected to the Internet and therefore are not usable within an AR system.  These sensors are also often proprietary. 

Companies like Pachube aim to change this by supplying sensors that can post data to the Internet (if you’re not familiar with why this is important read: Pachube, Patching the Planet: Interview with Usman Haque), therefore, giving information to the general public that can be put to good use.  However, either sensors must be put into place or environments tracked (Extended Enviroments Markup Language (EEML)). 

The placement of sensors or accessing environments creates a barrier to widespread implementation.  Creative workarounds can be used to collect information without dedicated sensors.  Crowdsourcing information can fill in the gaps, though the information might be messy, the wisdom of the crowd can transform it to be a usable source.

The breakdown of Reality Recognition into three parts with three levels gives it an easy scoring method.  Each section is worth three points:

Non-moving structural objects – High (3), Medium (2), Low (1) or None (0). 

Moving objects or people – High (3), Medium, (2), Low (1) or None (0).

Information – High (3), Medium (2), Low (1) or None (0). 

Since I’ve scaled it from one to ten, all these scores are added to the base score of one. 

The recently released Cyclopedia would score a two on the scale since it doesn’t understand moving or informational data and only scores as a low on the structural side.  This concept AR Playbox video below would score a three on the Reality Recognition scale because it would have to know general structural objects within the game area and also have to keep track of a moving person. 

Hopefully this more nuanced scale can help visualize the level of augmented reality and suggest areas that need refinement as we move towards an integrated Web 3.0 world.  If you have agreements, disagreements or suggestions for the scale, please leave a comment.

The last few months, we bloggers have decried the gimmicky nature of most augmented reality marketing campaigns.  Mostly, the problem lies in their clunky implementations of the technology.  Printing out markers and holding them up to the screen are as old as watching LOST or wearing Crocs. 

Successful usage of the technology relies on the user to forget there’s even some technical jargon behind it all.  Did my do-hickey connect with the thing-a-mabob?  Who cares?  Give me instant gratification. 

While I don’t have a single inkling to watch CW’s Vampire Diaries (I haven’t read the Twilight series either), I do think Inwindow Outdoor’s implementation of augmented reality is brilliant.    None of the people interacting with the Storescape window had even heard of augmented reality, and they didn’t need to, they were amazed by the screens ability to make them invisible.  Though I can’t for the life of me think of why these people would translate this impressive Storescape experience into watching a TV show, but I guess failure to understand cause and effect is what marketers rely on. 

Good product design is hard to create.

Interacting with the product should be effortless, engaging and should stimulate the user to think, “What have I been doing all these years without this product?”  When products are created with the end-user in mind, then problems are solved without the user ever having to ask. 

This should be the same for Augmented Reality applications.  We’re currently moving out of the R&D University phase of AR development where people were just happy to see proof of concept.  Now it’s got to appeal to a broader market that’s not going to put up with awkward interfaces and uneven usage. 

Imagine if car doors were like the General Lee from Duke’s of Hazard and you had to jump through the window to get in.

Imagine if you had to print out a piece of paper every time you wanted to watch a DVD.

Imagine if you only had two rows of text available on your emails and had to scroll up and down to read. 

The examples sound crazy, but it’s what we ask our customers to do sometimes.  We get caught in the trap of thinking, “If our customer only does A, then it’ll work just fine.”  But a customer isn’t paying for a work-around, they’re paying for an experience.  And if they experience is frustration, they’ll quickly discard your product (see this recent post from Ori about an AR product that failed customers as an example to the hazards of ignoring your customers needs.)

I get lots of experience watching customers use equipment in my day job as an engineering manager at a Toyota plant.  Suprisingly, the customers I speak of aren’t the people buying the cars.  They’re the team members running the equipment to make the cars.  I spend time watching our team members trying to use our equipment and the one thing I’ve learned time and time again is, if it’s hard to use, then they won’t use it.  Even if its “better.”

What does that mean for Augmented Reality?  Look back up to my second example.  Does it look familar?  Instead of printing out a piece of paper for a DVD, we’re asking them to print out a piece of paper to view augmented reality.  If AR is really going to take off, its going to need to be effortless, engaging and make people think, “How did we ever survive before Augmented Reality?”

Questions we need to consider if we want to make it an AR world:

Do printed markers really have a place in a big consumer market?

Will we be spammed with useless geospatial information once the market is mature?

How can my smartphone know the kind of information I want to display?

Should I have to hold my smartphone up everywhere while I’m walking?

Will the software be slow and cumbersome?

Isn’t the screen too small to display all the necessary information?

Will I have to scroll through many layers to find what I want?

Will I need to rearrange my living room to get my AR console to work right?

If you can remove the barriers to consumers using and enjoying your products, solving their problems before they even know they exist, then you’ll have made a great product.  The rewards for making a great product that customers can connect with are huge, but the punishment for failing their expectations are swift and merciless.

It’s all up to you.

Zugara brings us the next installment of AR shopping.  The company is an advertising agency that uses interactive marketing.  They have an impressive client list including the company I work for, Toyota. 

Their new AR product – The Webcam Social Shopper uses AR markers with motion capture to help the shopper “try-on” different clothing.  I like the way you only have to use the AR marker to set up the 3D plane, then once it captures the location, you can go markerless.  Also, it uses motion capture so you can scroll through the clothing choices without having to run back to your computer each time.  

The markerless and motion capture scrolling gives the Webcam Social Shopper an easy-to-use feel that I think is important for any product, especially AR.  The more steps it involves, the less chance people will bother to use it. 

[Via Disrpution]

If you’ve ever been skiing, then you know part of your time on the lift is spent figuring out where you are on the mountain, where you’re going to go next and hoping you don’t lose the rest of your group on the way there. 

The demo shown in this video will help paint the ski trails with correct difficulty level on them, show you the rest areas and lodging–and the most important feature–show the other members of your group (assuming they have the software) in their correct location on the mountain.

Hopefully in the final version it looks as nice and has all the functions shown , because I believe this would be a must-have app for ski trips.

I’d like to take a moment and review the AR happenings around the net in the last month about augmented reality.

* Not one, but two AR frameworks are being released – Layar and Gamaray; both for Google’s Android.  Augmented Times and Games Alfresco scooped both. 

* News on a number of handheld AR Games: Ghostwire and Invizimals.

* Lots of conference news in the last month.  Thankfully, the eyes on the scene, Ori Inbar, has brought back tons of interesting inside information: IGC East, E3 09, Where 2.0 and reflections on the GDC. 

* Interesting posts from a variety of blogs: AR Hype Cycle, Metaio’s AR Software Demo and High Visual Quality AR Demonstration.  I’m certain I’m missing lots of other good posts around the AR blogs.  If I missed you here, send me a line and I’ll add it.

* Many AR marketing campaigns (I’m not going to bother linking them): Papa Johns, Star Trek, Eminem, BMW, Toyota, Night at the Museum 2 and another half-dozen I’ve forgotten already.

* On my own site, my most popular posts have been: The Path To Augmented Vision, Creating the Cloud and How to Automate the Digitization of the World.  The last one is from March, but it still continues to be my top post. 

* Lastly, I’ve started a weekly AR comic called Augment This! and posted an AR short story called There Is No Truth In The Digital Sea.  The short story takes place about twenty years from now when augmented vision is commonplace.

Whew.  It certainly seems like activity in the AR world has been picking up.  I think the rest of 2009, AR will continue to grow and 2010 will be the real breakout year as good AR products will hit the market and start catching consumers eyes and wallets.

I’m not going to go into too much detail, because Rouli gives us an excellent run down on the Augmented Times, but I think this is an important step in “creating the cloud“.  Who knows, Layar could be the early version of Netscape, except for an AR world. 

Good job crew at SPRXMobile!