A group of students at the National University of Singapore has developed an AR game similar to Sim City using VRML and ARToolKit.

Virtual Sandbox is an educational software targeted at kids between the ages of 4-6. It provides children with an interactive learning environment for the learning of English vocabulary. By combining the concepts of Sim City, Flash Cards and Augmented Reality technology, we demonstrate how the bene?ts of both tangible and non-tangible interactions can be merged to create a fun, game-like learning experience for young children.

As a University project it gets high marks.  The controls wouldn’t be suitable for an older age group since using a PlayStation or Wii controller is more natural to kids these days.  But the younger crowd is more willing to move things around with their hands.  These types of games will need to move to gesture based controls like the EyePet  if they want to be commercialized. 

With the iPhone OS 3.1 launch set to debut next month, I wanted to explore the features that are going to help make some AR apps succeed over others.  We’re going to be seeing a spike of releases since many have been working towards this moment waiting for the iPhone to become AR enabled, so its a good time to get those apps in tip-top shape. 

And while I’m personally smartphone agnostic, these are features that should be present on any app, no matter what the platform.  But the OS3.1 release will connect the hype of AR with the hype of the iPhone, so that’s why we need to talk about app features now.  Call it Hype2.0 if you’d like.

And now for the features:

1 – Solutions Not Gimmicks

The best products and services are solutions to a problem.  Going to your webcam to open up a 3D version of the product has lost its luster as a tired gimmick.  Find a reason that augmented reality can be useful for your customers.  And if you’re a car maker looking to use AR, listen to Rouli. 

Don’t be like Always, the feminine hygiene company, please…

2 – Social Connections

The reason these social connectors like Twitter and Facebook work is because they allow us to be nosy neighbors seeing what next person is doing.  The term “Keeping up with the Jonses” applies here.  Give your app a way to connect other people in creative and interesting ways.  Even if its seeing the high score along with a picture of you dodging a cannonball.

 3 – App Search

Google dominates the search wars because they’re the best at it.  Many functions of AR require pulling information from the data sphere and if it gets hijacked by spammers then the app will quickly lose its purpose.  No one wants to pull up a restaurant layer to find a nice place to eat in a new town and get a porn layer instead while your three smiling children peer over your shoulder. 

4 – Alacrity Wins

I could say apps need to be quick, but they need to be more than that.  The Webster’s definition of alacrity is “promptness in response” or “cheerful readiness.”  Apps should anticipate what the user wants and get it quickly.  If it takes 10 seconds for the camera to identify where its at so you can decide which way to walk, no one is going to use it.  If the processor is too slow, move your heavy lifting to servers off the smartphone.  Life moves quickly, your app should move at the same pace.  

5 – Unique Style

If you want your business to grow, your style should be tasteful and easily recognizable.  When people see your products they should instantly know who created it.  Already the Layar symbol has stuck into my head as a good product identifier. 

6 – Not Another Vista!

If I let my kids cook dinner for themselves they make cereal.  My wife will take six hours to make an exquisite dinner that uses every dish in the house, including a few of our neighbors.  The kids eat cereal because its easy to do.  My kids are your users.  They’re not going to make Chateaubriand no matter how good it tastes.  So make your apps simple to use. 

7 - Save the Trees, Please

Please don’t make me print out another marker to see your version of augmented reality.  We might be able to say that AR is a green product, but for all the papervision markers we’ve had to print out.   

8 – Now With Less Spam

 For the AR apps that allow user added content, we need to make sure we’re not besieged with spammers like the regular Internet.  While this might be an impossible request, at least consider the spammers in your product design. 

9 – Tell Me A Story

Augmented reality was built by programmers, but it needs artists to grow.  Don’t forget to tell me a story and engage my emotions.  AR is a wonderful new medium with potential to tell stories in ways never told before.  The story can exist all around us or even in our living room. 

10 – FUN!!!

Now matter what the purpose of your app, even if its a serious one like saving the rain forest.  Don’t forget to make it fun (and maybe include a little magic).

OneZeroThrice has released ARtisan, an augmented reality library for developers.  The goal of ARtisan is stated as:

ARtisan is a front end and management system for FLARToolkit and Papervision3D. The goal of ARtisan is to facilitate developers in creating rich, browser based augmented reality environments, without having to know the inner workings of marker detection.

Currently they have five examples up on their site including the source code and a way to interact with the example on their site.

The goal of the site is commendable because it will allow latecomers to the AR scene a chance to catch up quickly with their examples and coding for FLARToolkit and Papervision3D.

Javad sent me a note that his talking head has been updated.  In relation to AR, the eyes move as the marker is moved to give the talking head a more realistic interaction. 

I’m not sure where Javad is going with this project, but I could see interesting applications when combined with a service like Ask.com or in an interactive console game with voice recognition. 

Keep up the good work!

Using a mix of programs including FLARToolkit for the AR portion, Javad has created a talking head that you can interact with.  Javad does a much better job of explaining the programs he used to make it happen:

What takes place is that we have an AIR client (built using Cairngorm) communicating with a Java server side using Remote Objects over BlazeDS. The text is sent to the Java server application using remote objects where a text response is generated using AIML and a Java chatbot framework. This text response is passed to a text to speech (TTS) socket server to generate both an mp3 byte array and something called MBROLA input format. MBROLA input format is a stream of text symbols (phonemes) together with duration in milliseconds, that represent visemes (mouth shapes).

The whole lot is packaged and sent back over the wire via BlazeDS where we have an Augmented Reality Viewer create as an Advanced Flex Visual Component (using Papervision3D and FLARToolkit). The model head was created in Maya and is an animated Collada with 13 different mouth shapes that have been mapped to the output received from the MBROLA stream.

To play the speech response, the mp3 byte array is written as a temporary file, read into a sound object and then played back. At the same time the MROLA stream has been parsed into an ArrayCollection of frames (for the model head) and durations and this is now iterated over in the handler method of a timer.

All this back end work results in an impressive demonstration shown in this video:

This talking head has a lot of potential applications from gaming to educational, though mostly it reminds me of Max Headroom. 

Development of AR in all its incarnations will come from a variety of sources.  Javad is showing his contribution through this admirable project.  Stop by his blog and say hello.

I’d like to take a moment and review the AR happenings around the net in the last month about augmented reality.

* Not one, but two AR frameworks are being released – Layar and Gamaray; both for Google’s Android.  Augmented Times and Games Alfresco scooped both. 

* News on a number of handheld AR Games: Ghostwire and Invizimals.

* Lots of conference news in the last month.  Thankfully, the eyes on the scene, Ori Inbar, has brought back tons of interesting inside information: IGC East, E3 09, Where 2.0 and reflections on the GDC. 

* Interesting posts from a variety of blogs: AR Hype Cycle, Metaio’s AR Software Demo and High Visual Quality AR Demonstration.  I’m certain I’m missing lots of other good posts around the AR blogs.  If I missed you here, send me a line and I’ll add it.

* Many AR marketing campaigns (I’m not going to bother linking them): Papa Johns, Star Trek, Eminem, BMW, Toyota, Night at the Museum 2 and another half-dozen I’ve forgotten already.

* On my own site, my most popular posts have been: The Path To Augmented Vision, Creating the Cloud and How to Automate the Digitization of the World.  The last one is from March, but it still continues to be my top post. 

* Lastly, I’ve started a weekly AR comic called Augment This! and posted an AR short story called There Is No Truth In The Digital Sea.  The short story takes place about twenty years from now when augmented vision is commonplace.

Whew.  It certainly seems like activity in the AR world has been picking up.  I think the rest of 2009, AR will continue to grow and 2010 will be the real breakout year as good AR products will hit the market and start catching consumers eyes and wallets.

If you’ve ever bought a marble run, rollercoaster or any other type of plastic track that requires hours of set up and falls to pieces at the wrong touch, then you’ll appreciate this idea. 

Last week, my wife and kids set up a marble run in our living room.  It took hours to set up because the tracks kept falling over.  The bases were too light to hold up any track over a foot tall.  Once they had all the bases secured down with heavy books they had to contend a new problem–the ball jumping the tracks on tight turns. 

They got it working finally, but making any changes to it would have been frustrating so the result ended up to be pretty static and they took it down that night.  I thought it seemed like a perfect solution for a console AR game. 

The advantage of using a console (Wii, XBox, Playstation3, etc) is that the structure of AR family entertainment center is already in place.  With a big screen TV, a wide open space, and camera, the living room could be converted into an exciting game center.  In much the same way that the Wii revolutionized interaction with the console games with gesture based gaming, AR could up the ante by providing a visual context for the games. 

Using the simple idea of the marble run, operators could place tracks in their living room using the controller.  The program could snap them together creaing fantasical designs.  If you wanted to learn more about physics, you could have a “realistic” version that shot the marble off if you made the tracks wrong. 

Roller coaster design games like Roller Coaster Tycoon have been around for years on the PC, but using AR could reinvigorate them by making the game come to life in the living room.   The use of AR should be brought to market in the same way as the Wii.  They didn’t have better graphics, but instead showed how getting people physically involved with the games got people interested, even people that had never played games before.

Tim Verbelen and Pieter Dhondt have brought us the results of their MSc thesis.  In the video they take the concept of a FPS (First Person Shooter) like Doom or Quake and apply it to the real world.  Using materials available to them like papervision and Wii controllers, they were able to put on an impressive demonstration of their game. 

While its not ready to become a game you can buy at your local game store, it certainly showcases what can be done even now with a little creativity and hard work.  Kudos on a job well done. 

It’s official.  I can’t get excited about AR marketing campaigns or AR products that don’t do anything.  I need an invisible creature on my desk that I can only see on my iPhone as much as I need a set of bowling pins. 

But I’m not going to sit here and complain about that.  Instead I’m going to give some suggestions for AR products I’d like to see in a series I’m going to do called–Steal This Idea!

This week I want to talk about a slice of the gaming market that is absolutely ripe for Augmented Reality–miniature gaming.  If you’re not familiar with miniature gaming, then I’ll break it down for you.  Miniature games are all about tactical combat on a table sized board with terrain. 

The fun of miniature gaming is standing around a table with a few of your close friends and outwitting them in tactical combat using insanely cool looking painted pewter figures.  I’ve played a variety of these games all the way back to Chainmail, the game that eventually spawned Dungeons and Dragons. 

The downside of these games is the high cost for the miniatures, poor portability of the equipment, painting the figures and arbitrating the rules.  These are problems augmented reality was meant to solve.  The style of play even works for marker AR because base plates are used to hold regiments of figures. 

I’d like to see a miniature game company pair with one of the big console game companies to create an AR miniature game.  The reason why this is a perfect fit is because most people with a console game like XBox, the Wii or Playstation will have a big screen to go along with their console.  A big screen is essential for seeing all the action on the table. 

Instead of having to buy a $600 painted Warhammer army, a high cost of entry, one could simply download an army to play.  The miniatures could be rendered in devilish detail down to the blood on the scythes of the skeletal horseman.  The console could arbitrate the rules in a convenient fashion, or if they preferred rolling dice, could read the dice thrown onto the table. 

Games could be played on a table near the big screen with all the details drawn onto the table using augmented reality.  This would keep the game active with opponents moving around the table rather than being stuck at a stationary screen like most computerized strategy games.  The actual resolution of the combat in gory detail would be a huge plus.  Armies could leap at each other, drenching each individual combattant in glorious battlelust. 

If these thoughts aren’t enough to get a fledgling AR game developer to jump into the fray, think about the demographic of the miniature gamer.  They’re willing to plunk down thousands of dollars for painted miniatures, terrain and battle tables.  They’d be happy to spend a tiny fraction of an army cost to purchase a new way to play an old game.

Augmented Vision will be available in 2015.

Am I prophetic?  Delusional?  Or merely guessing?  How does an deluded prophetic hand-waving guess sound?

The reality is the development of complex systems like Augmented Vision will take something more than putting the various technologies together.  AV will require a change in the zeitgeist similar to the iPhone.  But that’s not what I’m here to talk about.

While the magic moment–the tipping point–will take some unknown trigger.  The technologies will have to be available to support it.  The iPhone couldn’t have existed five years ago, just like AV can’t exist right now. 

The first thing to ask is, what is Augmented Vision?  I will attempt to define the term, but others may differ with it.  That is okay, as I am only trying to place a target in space to draw an arrow to (or in this case, many arrows). 

Definition of Augmented Vision: an unobtrusive self-contained human based system that creates an augmented reality experience allowing the user to interact with any object in the populated world.  Let’s break that down into its pieces. 

1 – Unobtrusive self-contained: the ability for the devices to be fashionable, easy-to-wear and comfortable.

2 – Human oriented: centered around the everyday human experience.

3 – Creates an AR experience: the cloud is a mature system overflowing with content.

4 – Interact with any object in the populated world: in our modern surroundings, anything can be identified, located and learned from.

I’m not speaking of an AV experience that makes reality and virtual difficult to decipher like Denno Coil.  I’m thinking of AV as a tool to enhance the everyday living experiece just as the way other technologies have like the iPhone.

Computational Power – I’m using the specification difference from the iPhone 1.0 to 3.0 and considering Moore’s Law to generate a linear projection.  I think 4.8GHz would be plenty of processing power to perform most operations, and Rouli pointed out to me that hard-core algorithms will be computed in the cloud like SREngine and Alcatel Lucent’s initiative.   So computational power won’t be the limiting factor.

Vision Systems – Not much to go on here except the release of the AV920 from Vuzix in the fall.  Looking at the cellphone development cycle from the last ten years, I’m thinking lightweight, fashionable and comfortable AV glasses will be available to the masses in five years. 

Control systems – Talk, type, touch and think.  Typing and touch are the current system.  Think is too far off to be realistic for a human based system.  This leaves talking and “air-touch” as the probably control systems.  Much of the technology is already known, so control systems won’t limit AV. 

Software – This is the biggest unknown.  What makes up the bag of tricks required to make an AV platform similar to an iPhone?  Object recognition, outdoor markerless, perfect occlusion, non-rigid surfaces, optimizing frames-per-second, geolayers, etc.  I placed some items on the progression, but its hard to say which ones will be needed and how they fit together.  The tools eventually needed will depend on the creativity of the manufacturer.  I could speculate further but this is a question better answered by someone else. 

While this post is mostly speculation based on the available research information and the limited commercial products on the market, I think it is a useful exercise to see the direction the technology is headed.  Looking at the components required and seeing the gaps in development, an entrepreneur might use the opportunity to fill in the gap with the right product.

Once created, the basic Augmented Vision will be more like Terminator Vision, but it will create a platform to launch from.  Will it grow until it reaches Augmented Vision that blurs reality and the virtual like Denno Coil or the Digital Sea?  I might not find out in my lifetime, but until then, I’d be happy with an AV that gives me good hands-free directions to the nearest pub for a pint of Guinness.