WIP 007: AR Explorations

I am interested in the intersection of the physical and digital worlds. 3D printing and digital fabrication empowers students to learn by creating and remixing digital designs and then producing physical models. Augmented Reality – AR overlays digital 3D models on the physical world. I first became interested in this possibility in June 2018 when Apple and Pixar announced the new USDZ file format for AR content.

Ideally, sharing a 3D model/AR scene would be as simple as sharing a photo or video. Social media platforms, websites, productivity tools  and digital devices should handle these models seamlessly. There should be countless apps that allow the creation and remixing of 3D models in AR. Currently, most AR experiences are locked into specific hardware and apps but there are a few new methods that allow app-free AR experiences on IOS/Android devices that provide native support for AR.

Here are a few of my explorations. If you have ideas on easier and more powerful ways to publish AR, please leave a comment or contact me on Twitter @DesignMakeTeach.

Model-Viewer Web Component – HTML: I’m starting with the best example first. The model-viewer web component by Google allows the display of interactive 3D models in browser with AR available on IOS and Android mobile devices. This is the best of all worlds because it doesn’t require an app to be installed and works in browsers on all devices and in AR on newer mobile devices. WordPress.com doesn’t support using HTML code with scripts so I’ve created a separate site to show these examples. Go check it out then come back to read the workflow.

CLICK THIS LINK to go to the super cool example! https://sites.google.com/view/ar-designmaketeach/home

-The model-viewer code to insert models in to your website is available at https://modelviewer.dev/

Workflow for examples
-Download glTF/GLB file (Smithsonian and Tinkercad were direct download. NIH 3D print was converted from X3D file using Sketchfab)
-Import glTF/GLB file into Adobe Aero project on IOS
-Export As USDZ file
-Save glTF/GLB and USDZ file to Dropbox
-Get Dropbox link to each file
-Change Dropbox file links, replacing http://www.dropbox.com portion of link with dl.dropboxusercontent.com
-Publish model on Google Sites using the model-viewer code inside an Embed box.
-Modify the model-viewer tag code by inserting, auto-rotate ar magic-leap
quick-look-browsers=”safari chrome”

I’ve also been able to use GitHub and Glitch to publish the AR files but the Google Sites option seems simpler.

AR Quick Look- USDZ Model for IOS – HTML: This example uses the following html code to show a static image which can be clicked on newer IOS devices to open AR Quick Look view. This should support files in .usdz or .reality format.

<div>
<a rel=”ar” href=”/assets/models/my-model.usdz”>
<img src=”/assets/models/my-model-thumbnail.jpg”>
</a>
</div>

Click on the photo using a newer IOS device to view in AR.

I created this example using the following workflow.
-Export TinkerCAD model in GLTF (.glb) file format
-Open file in Adobe Aero
-Export as .usdz file from Adobe Aero
-Save .usdz file on web
-Add HTML code to website

Adobe Aero for IOS: Build and share AR scenes using 2D and 3D files. Can share link to for others to open in model in Adobe Aero app. Convert files to .usdz format. link

Here is an Adobe Aero model example showing 2D and 3D content. This should open the model in the Adobe Aero app if you have it installed. https://adobeaero.app.link/kbLpgk6Ht4  

Reality Converter for Mac: This beta software is available from the Apple Developer page. “The new Reality Converter app makes it easy to convert, view, and customize USDZ 3D objects on Mac. Simply drag-and-drop common 3D file formats, such as .obj, .gltf and .usd, to view the converted USDZ result, customize material properties with your own textures, and edit file metadata.” link

Reality Composer for IOS and Mac: Create AR scenes using imported USDZ files. link

Sketchfab App: Sketchfab is a web platform for publishing and finding 3D models. It has a nice embed functionality to add 3D models to web sites. The Sketchfab app supports AR view for IOS and Android. link

TinkerCAD App on iPad: If you have the TinkerCAD app on iPad you can explore your models in TinkerCAD’s AR viewer. Open the design and then click the AR Viewer button to explore the model. link

Morphi App on iPad: Morphi, a 3D modeling and design app, was the first on IOS that I saw supporting AR view of 3D models. link