Developing Lumin Runtime Apps - ARIA 2020

Developing Lumin Runtime Apps for Magic Leap

• The ability to have multiple applications open

• Eg: A clock and a chessboard

• Game Dev engines are designed to run one thing at one time

[Scene Based IA]

Is like a Scene Graph

The method that engineers are using

Is similar to IA that designers should use

But instead of Node - we should define what the content is

Input Events

Environment Events

Multi-user

Send the scene graph to the server

(Does that make sense?)

In theory - run everything in the cloud

Bigger, Faster, and Out of Hand - Steve Feiner

Talk 1 

Increasing FOV in displays

BIGGER

Sometimes a bigger field of view is more important than a wider field of view

Projection on the environment is considered a Gold standard - perfect registration 

Width is the metric people use - people want wider FOV

But there are many applications when height is the issue

Eg: - walking downstairs with low vision - not a context when using a cane makes sense

FASTER

Decrease latency

The time btwn stimulus (cause) and response (effect)

• Eg: motion to photons 

  —  Motion can be head, hands world

Most  electronics in our environment are more than 100 milliseconds delay

Some are 10 milliseconds 

Perceptible Latency (university of Toronto) 1ms-latency 2D touch screen  2013

Around 6 milliseconds is noticeable (in a random group of people - not a group selected with high reflexes - eg athletes, race car drivers, pro gamers) 

OUT OF HANDS

Context without hands

Eg Hands-free AR for vascular intervention

• Both hands are busy with a catheter - 

• Using Headpose & voice

[See video]

• Scaling and rotating based on head pose

• Increase rotation and rate of rotation by moving head away from virtual object

• Change location by head pose

This is also important for accessibility 

Importance in many domains of not using hands at all,

Columbia Engineering

The Fu Foundation School of Engineering and Applied Science

Presented at MIT Media Lab, Cambridge Mass

AR in Action 2020

Non-profit #AR org bringing people from around the world. They charge nothing for the booths here One reason it’s called the anti-CES.

Explicitly thinking about implications for contextual computing - next computing platform.

When Bill Gates wrote The Road Ahead He didn’t even put the word intent in there He had to add it later .

AR is harder than VR We haven’t totally figured it out in terms of enterprise and consumer

Should be a place where AR innovators can meet and connect.

All of these talks are online and free

This is the 4th annual ARIA.

There was a 5th that happened in 2017 - at Javits Center in New York City.
If you are in New York and interested in AR. If ARIA is back - skip work and attend ARIA.

First head mounted display - AR. 52 years ago, a mile from here @medialab by Ivan Sutherland pic.twitter.com/5LuvIAlXxW

— Sam Brewton 🚀 AR in Action @ MIT Media Lab (@ironandsilk) February 12, 2020

Versailles VR by Google

https://artsandculture.google.com/project/versailles

Hat tip - Tom Emrich

https://www.linkedin.com/feed/update/urn:li:activity:6633147563975426049/

MIT Hack Reality 2020 hackathon

Favorite Projects from the MIT Reality Hackathon

Top things I learned plus more info and imagery to come

MIT Reality Hack 2020: Hack to the future at the coolest XR hackathon! - Devpost

Accessibility Toolkit For Unity | Devpost

GhostBustXR | Devpost

#³ | Devpost (hashtag cubed)

NutriMonster | Devpost

Spell Bound | Devpost

VSiBL | Devpost

AirSpace | Devpost

Accessibility Toolkit For Unity | Devpost

draft360 | Devpost

- 360° Image on glitch

Harvard Graduate School of Design

Reviewing portfolios last night at Harvard University Graduate School of Design with design leadership from Wayfair

I remember architects would come to hashtag#IxDA looking to break into User Experience. Now, architects can leapfrog to the next generation of digital product design with their 3D and spatial output while maintaining a human-centered approach.

Landscape architecture intervention by Xiaoji Zhou xiaojzhou@gsd.hardvard.edu

Brand installation simulated in Unity by Shi Tang www.tangs23.com stang@gsd.harvard.edu

Apple [AR]T Walks

MSFT Azure Cognitive Speech Services

Focals by North

I used Amazon Alexa on the Focals, asked for a cafe nearby and requested directions.
It’s kind of like a smartwatch on your face.

Tech:
_ Optics: Laser
_ 15 degree viewing area
_ 300 x 300px
_ Monocular

i/o
_ Microphone
_ Voice to Text with Amazon Alex
_ Speaker
_ No Camera


Experience:
The Loop (the ring) - an input device like a joystick on a ring.

Customer service - impeccable.
You get you face scanned.

”Hard to see in daylight
The Ring input device is water resistant.

Mapping Service is Mapbox

Showed “lost internet connection”
[Issues with mapping - does not provide a very accurate position of your location.]

Needed to create a different number to receive text notifications
Text messages are not sent to the Focals, they are sent to North’s server and then North’s server sends a message to the Focals.

Can only order Uber X.
I grew to having enjoy notifications.
They feel like the future.”

You can check them out in Toronto and Brooklyn. The environment, customer service and even the response time from the Focals team on social media is really impressive.
Learn more here https://www.bynorth.com/focals

Visit in BK here

NORTH COBBLE HILL

178 Court St | Brooklyn, NY 11201

A visual positioning system of computer vision and machine learning.

Anything that’s been documented by Street View, like buildings but not trees because branches get cut and they lose leaves.