Hello (AR) World

Reality, Augmented

You sit on a train with your cool-shade filtering out bright light, earphones on, rain-pouring-down white-noise helps drawing out this buzzing noise so at least you can concentrate on what you are reading - the news that your government announced GDP growth of 0.3%. Your friend said that you are his best mate and asked to borrow some money; your girlfriend said that she really like the green necktie you wear today. Your boss said that you were her best employee and asked if you could do OT this Saturday. You hate lies. But, all these are alright - you are telling yourself that they were augmented. What's real? What's not? Never mind that, let's alter our perceptions a little bit more, you picked up a book on Augmented Reality and search internet on AR. 

Hardware & Software Maturity

Apple introduces Augmented Reality back in WWDC 2017, with ARKit 1.0 and iOS 11, signifies hardware and software maturity to support altering our reality in realtime - with device's camera, Apple A9 CPU, GPU, and wide array of sensors, working in concert to interact, analyse, and produce realistic virtual content on top of real-world scenes.

Jump onto the bandwagon

Fast forward to 2019, Apple released ARKit 2.0, iOS 12, and the all mighty Xcode 10.2. The library is more mature, the iOS is ever more efficient, the tools allow creating AR applications seamlessly - like magic. AR becomes the talk of the town. I thought to myself ... it is now, or never.

But first, these are the things I need to get before I can get start:

  • iPhone from 6s or newer, and/or iPad from 2017 or newer
  • iOS 11 or newer
  • Mac from 2009 
  • Xcode 9 or newer
I was lucky to have these items available and all posting here are based on them:
  • iPhone X & iPad 2018 (not the Pro - I didn't have $$$)
  • iOS 12.2
  • Macbook 2016 (1.2 GHz, core m5, 8GB RAM)
  • Xcode 10.2.1 (don't get 10.2 - it's buggy)

The Almighty Xcode

I love the Xcode. I really do. I love the way how Xcode broke all my plugins every time they updated to the new versions and they do make sure that it happens very *very* frequently. But, that is for another post. Obsolency conspiracy, anyone? No. I mean "yes" - I really do love the Xcode. I was awe in 2017 when Apple put SpriteKit builder in Xcode 9. I have to same awe-struck feeling now with SceneKit's Scene Editor that support 3D and AR authoring.
The best place to get yourself familiar with Xcode's SceneKit Scene Editor is - with the Xcode! From Editor menu, you can select `SceneKit Editor Help` and have a good read there.

KISS - Keep It Swift and Simple

It is true that with the Xcode's default AR-app template, you can create a good looking AR app with almost no additional line of code. Fair dinkum!

In Xcode, create a new project and use the default Augmented Reality App template. Follow through and specify details for your new project, you should end up with project structure like this.

Assume that you are familiar with Xcode and iOS development in general, AR project is the simplest single-view iOS app - with an additional of the scene asset file - the `art.scnasset` file

Playing with 3D Objects

Scene asset file contains all the 3D objects. Each tab in Scene Editor provides with very educational details. 

One particular of interest is the Node Inspector. With Node Inspector you can see:

  • object's position on (x, y, z) planes
  • Euler (x, y, z)
    • Rotation around x-axis - pitch
    • Rotation around y-axis - roll
    • Rotation around z-axis - yaw
  • object's scale (x, y, z)
Note that all units are in meter (thanks God! Something non-american!), for example, 'Bounding Box' measurements gives you the size of the object in real world, in the above example that 48.96 x 17.22 x 46.64 meter! That's huge! Try build the project and run on an iPhone. You will need a big back yard to fit that plane there! My front yard is small, so I need to scale it down a bit.

To fit that plane in my front yard, I scaled it down with 0.02 factor (on all x, y, z axises) - that gives the plane the size of about 1 x 0.3 x 1 meter. Don't worry, the Scene Inspector actually calculates the final *real-world* metrics for you.

Now, that plane fits nicely in my front yard.

No Line Of Code - Almost

Let's take a look at the source code. If you are familiar with iOS development, you will see that the codes are at the minimum a single-view app. Taking a look at `Main.storyboard`, you will see a single ARSCNView embedded in the main view of the ViewController. All 3D objects and models are loaded from `art.scnassets/ship.scn` file in `viewDidLoad()` as show below.

I have added sceneView.debugOptions as extra lines of code in to make the project a little bit more interesting. When I tell you that no extra lines of code needed, I didn't lie - just *augmented* truth 😅

  override func viewDidLoad() {

    // Set the view's delegate
    sceneView.delegate = self

    // Show statistics such as fps and timing information
    sceneView.showsStatistics = true

    // Create a new scene
    let scene = SCNScene(named: "art.scnassets/ship.scn")!

    // Set the scene to the view
    sceneView.scene = scene

    sceneView.debugOptions = [

Build the project and run on a device. Have a walk around your new shinny 3D plane. Walk to some where else or other rooms. And when you come back to this spot, notice your 3D plane is still there! Amazing! All this was done as part of real-world objects tracking provided for you automatically by a few lines of codes.

  override func viewWillAppear(_ animated: Bool) {

    // Create a session configuration
    let configuration = ARWorldTrackingConfiguration()

    // Run the view's session

Awesome! This is fantastic! I showed my app to my son and went ballistic with joy! He ran around and inspected the shiny plane from all angles. He can't wait for me to put more cool things there - robots, spaceships, cool sport cars, trees, and so on.

Ok! How do I get or make these cool 3D models? Where to find them? How to put them into the scenes? Little beads of sweat formed on my head.

"Yes, son. That's easy. Let daddy work on it", I replied.

Further Reading

  1. iOS 11 - Wikipedia
  2. ARKit - Apple Developer
  3. How to Get Started with Apple's ARKit

No comments:

Post a Comment