Flash Augmented Reality on iPhone and Android ? Not so fast..

May 20th, 2011 by Slav

This is one of those articles where title pretty much says it all.

Now with AIR 2.6 iOS camera support and the unrestricted camera feed access to developers in iOS4, we can finaly build some cool AR apps on the mobiles right ?. Well… in theory at least!

The Setup :
To see how it would perform, I made a quick AR demo using Flar toolkit 2.5.5 for motion tracking and pattern recognision and Papervision 2.1 for the 3D scene. Scene consist of a simple flat shaded 3Ds model,  plus 4 interactive buttons. The whole scene has less then 170 polygons. A very lightweight scene with very low poly count. To make the conditions fair for both devices I kept the AR scene dimensions the same (hence it looks smaller on HTC Desire HD). Both apps use gpu rendering.

I exported the app into an iPhone format using latest available AIR SDK – 2.6. For comparison I also converted the flash project into Android format. I tested the iPhone version on iPhone 4 and Android version on HTC Desire HD.

The demo has 2 buttons that enable the user to turn off the motion tracking/pattern recognition – (FLAR ON/OFF), and scene 3D rendering. By turning off the 3D rendering, and seeing the changes in frame rate,  we can see how much resources are consumed only by the rendering of the 3D scene. By turning off the 3D rendering we can see how much resources are spent on the motion tracking.

Here’s the quick video showing the iPhone, and Android version

Framerate results comparison:

(Swf had frame rate set to 25fps)

And here are the lessons learned :

Lesson 1: Augmented reality can be implemented on mobile devices via flash conversion, just not in any usable manner

Although app will still run and function, the poor frame renders the application unusable. The problem is not just visual – stuttering screen playback, it’s also functional. If you have interactive elements in the scene such as buttons, low frame rate will make a click / touch detection difficult and user would have to try multiple times for a touch event to register.

Lesson 2: iOS conversion performance is superior to Android one.
Although both devices run on a similar hardware (both have 1GHz processor) and the apps were converted from the identical swf file – Android performance was on average worse. Also, while framerate on the iPhone seemed to be pretty stable, on the Android it was generally erratic and fluctuating.

Why is this happening?

I personally think it’s due to the way the apps are being converted and run on the device. While iPhone app is generated by converting the swf file into a native app, android app is being converted into an android air app compatible file and run by the Air runtime wrapper that has to be separately installed on the device.
That , in my opinion , creates a performance decrease , as it requires an extra hardware abstraction layer between the code and the device hardware. Think of it as flash file being run inside a flash plugin in the browser, versus OS native app. By design, swf app will never achieve the same performance as the native one.

The question here is , why does Adobe approaches Android conversion in this way. As for the iOS, it is quite clear why the iPhone apps are converted to native format. Apple wouldn’t let them do the Air wrapper approach. For the Android , they don’t need to however, yet they choose to do it. From the marketing point of view , it makes perfect sense of course. In the same way the spread of flash plugin helps to spread the entire platform and the tools (mainly the creative suite).
From the developers perspective it’s not that great though.

It means that every Android user , in order to run your app, will have to download AIR for android runtime. Although this helps to keep the file size relatively small (android version 451Kb vs 5.3MB iPhone version) the air runtime itself has 17 MB!
I would really like to hear justification from Adobe, as they were really conscious about increasing the file size of flash plugin (still less then 3MB) when almost everybody has broadband at home, yet they’re OK with forcing people to download 17MB on 2G/3G mobiles? I think they might be shooting themselves (and developers ) in the foot here.

The extra AIR Runtime file size issue is bad enough, but if you add an inferior and erratic performance to the mix there is really no reason to prefer conversion to air vs native app.

I am aware that performance in AIR 2.7 is better (there’s no Android Air runtime available at this moment so can’t test this personally) and with flash 11 and molehill we might see even better increase when it comes to GPU rendering.

However; there will always be applications that rely heavily on the cpu and for those, native app conversion will be always superior option.

Posted in AIR, Android, AS3, Augmented Reality, flash | 34 Comments »

Augmented reality helicopter

July 13th, 2009 by Slav

Augmented reality is slowly becoming perhaps somewhat of a bevel effect/lensflare among the technology enthusiasts, but actually it’s only beginning to creep into consciousness of the “marketing people” (as I call them) so I though it’s high time I joined the ranks and pop my AR cherry. : – )

This demo is based on excellent ar tutorial by Lee Brimelow, you should definitely check it out if you want to play with AR.

One thing I discovered, the marker pickup up by FLARE toolkit can be quite tricky at times and it really helps if you do apply a threshold filter to an input bitmap data.

I added basic flying controls to helicopter and a helipad plane to ground it on the surface..

AR Chopper

Demo was build with Adobe Flash Builder demo, but the most important class extends UIComponent, so should be easily transferable to Flex 3.

You can view the demo here and right click to view the source.

If you don’t have a web cam watch screencast here.

Posted in AS3, Augmented Reality, Flex 4, papervision | 12 Comments »