Even unseasoned smartphone users are likely to have interacted with augmented reality (AR) technology in the last year. In June 2016, Pokémon Go helped AR become mainstream. Just one year later, AR was touted as a future strategy for every technology company with a worldwide ambition. Offering rival technological frameworks, the “big tech” companies hope developers will create new experiences for their platform.

New augmented reality apps can guide you to your destination, but what technological framework will they make use of? Illustration: Omar Prestwich/Ståle Grut (Unsplash License)

Back to the future

While the technology itself emerged from experiments at Harvard in the early 1960s, the term augmented reality was first coined by Boeing researcher Tom Caudell. He was trying to superimpose diagrams and markings on the airplane factory floor in the early 1990s (Mann). Despite these early experiments, the technology did not see mainstream adoption until 2016. At that point, many big technology companies had already been working for some time on developing and commercializing the adjacent virtual reality (VR) technology. While VR need massive machine power and complex hardware today, AR turned out to be easier to use and commercialize. All thanks to powerful mobile phones equipped with a wide range of sensors and a well-established software distribution channel in form of app stores. This helped the game Pokémon Go become a massive global hit in the summer of 2016 (Addley). While mobile apps utilizing AR already existed, Pokémon GO was the phenomenon that “brought augmented reality to a mass audience” (Wingfield og Isaac).

Pokémon Go combined the GPS and camera in a smartphone to locate and give agency to players in a reality created inside the app (Sicart). Equipped with only their smartphone, users could navigate a parallel digital universe, where fantasy animals and other fictitious points of interest appeared on screen. They were connected to landmarks, parks and other public spaces in the real world. This stimulated simultaneous exploration of the user’s real and virtual environment. The whole experience was wrapped in the design of cult phenomenon Pokémon, upping the game to viral sensation. Soon large crowds could form in the real world due to a rare Pokémon figure appearing in the app. Within a month of release, reports of mobs, stampedes and mayhem were used in the news to describe the ensuing events.

War of platforms

Snapchat is one of the world’s most innovative and influential consumer technology companies (Manjoo). They are also one of the companies at the forefront of AR experimentation. In September 2016, they announced a new strategic direction. They were no longer a social company, but rather “a camera company”. Facebook followed suit, and by November its CEO Mark Zuckerberg said “[soon] we believe the camera will be the main way that we share” (Galloway). Snapchat early on used AR to let users apply masks to their selfies (Newton). At their annual developer conference in 2017, Facebook went further. The company made it clear they aimed to “platformize” AR technology (Zuckerberg).

The term platformization is commonly used to describe the platforms rise as a dominant infrastructural and economic model of the social web (Helmond 5). Facebook now applied this to AR as well. To do this, they presented a new framework called AR Studio. It would enable developers to create their own products, such as “animated frames, masks, and interactive effects that respond to motion” within Facebook’s platform (Constine, Facebook launches augmented reality Camera Effects developer platform). It led to predictions of AR becoming the next battleground for big tech (Vanian). While the technology itself was not new, the strategic role suddenly ascribed to it was both new and interesting.

Mark Zuckerberg presenting his vision for AR at Facebook’s developer conference F8 in April 2017 Photo: Anthony Quintano (CC BY 2.0)

Software frameworks, like AR Studio, usually consist of special tools and APIs made by companies or platforms for developers. Using frameworks, developers can avoid building an entirely new system for each product they want to release (Baldwin og Woodard). If a developer chooses e.g. Facebook’s framework, Facebook achieves a lock-in. This ties the developer to a platform, and is a principle goal of any platform builder (Plantin, Lagoze og Edwards 7). Facebook was not the only company that had this idea. In the case of AR, they soon met fierce competition from platform rivals Apple and Google.

For the users, accessing Facebooks AR functionality requires them to install an app on their mobile device. These devices are mainly smartphones running Android or iOS operating systems. It means that Facebook is at some mercy of platforms controlled by Apple or Google. Facebook has a long tradition of trying to escape the confinements of Android and iOS, e.g. by creating their own phone and even a type of Android operating system, but has not yet succeeded. This is likely one of the reasons why they are investing in emerging technologies like VR, hoping to one day dominate both the hardware and software side of a new technology (Isaac).

Battle of frameworks

Likely fuelled by the success of Pokémon Go, with its 750 million downloads in the last year, Apple and Google this summer contested Facebooks “platformization” of AR (Chan). Both announced that the technology would be implemented as a core part of their next generation of operating systems. Apple announced the framework “ARKit” for iOS, while Google presented “ARCore” for Android. These frameworks give developers direct access to advanced APIs and technologies within the operating systems, and “easily enables developers to add AR capabilities to their apps” (Slater). Platform builders seek to create such modular frameworks that are under central control, so they can benefit from the innovation made by a vast amount of independent developers (Plantin, Lagoze og Edwards 7).

AR is not a unique example of this. Apple, for instance, have developed similar frameworks for health monitoring and home automation, called HealthKit and HomeKit. To benefit of the developer’s innovations, the platforms need to entice them into choosing their framework. They all have somewhat different sales pitches. Googles tagline for AR Core hints to the large user base of their operating system: “Transform the future of work and play at Android scale” (Google). Apple talks about ARKit’s compatibility with existing developer tools and how their software and hardware goes together to create better experiences (Buerli). With their two billion monthly active users, Facebook emphasizes the social element. They highlight opportunities to use AR technology with “fans” and “artists” (Nowak og Spiller) (Facebook).

Both Apple and Google’s announcement of directly implementing AR in the operating system undercut Facebooks effort to create an AR platform of their own, “[s]ince every iOS app can easily plug in AR features” (Constine, Apple undercuts Facebook in the augmented reality platform war). Apple gave a good example of what this entail at their annual developer conference in June 2017. There they presented an improved version of Pokémon Go. It made use of the newly introduced ARKit, enabling a much more advanced gameplay for iOS users (Buerli).

Where it goes from here

A majority of the big tech companies – like Apple, Google, Facebook, Microsoft, Amazon and Snapchat – are currently working on new uses of AR and similar technologies. The smartphone is currently the place that mainstream AR manifest. However, both Microsoft and Google have for several years developed head-worn devices that enable AR without using a smartphone display. Researcher Babak A. Parviz conceived Google’s Glass and is now working on Amazons latest AR device (Confliffe). From his earlier research with combining smart contact lenses and augmented reality, he claims that “a contact-lens display could unlock whole new worlds of visual information, unfettered by the constraints of a physical display” (Parviz).

Others are more critical of the technology. Evgeny Morozov describe AR as “a fancy word term for infusing our everyday environment with smart technologies” (Morozov 20). He is a sceptic towards AR and how it was used in a Japanese recipe experiment. There, the user would get detailed instructions via AR about e.g. where the knife should be placed on the fish to cut it open. Morozov argue that large scale adoption of this technology will deprive humans the opportunity to fully master any skill. With a shrinking room for experimentation, there can be no human flourishing (Morozov 20). Artist Keiichi Matsuda has tried to envision such a future, where the whole world is enmeshed in augmented reality. In a short film called Hyper-Reality, he presents a future where physical and virtual realities have merged, and the city is “saturated in media” (Matsuda).

While AR looks simple today, Matsuda’s vision could become a reality. But it is not yet clear whose framework would dominate it. Perhaps today’s rivals should once again look to Pokémon Go. Recently the CTO of the company behind the game, Phil Keslin, said that using AR on a smartphone is unnatural. He also said stated that it makes people look like “a total doofus if they’re doing it for an extended period of time” (Perez). Given this is true, the previously mentioned frameworks could constitute a dead end.

Keslin’s description does not sound like a winning sales pitch for any framework hoping to dominate the AR space in the future.

Bibliography

Addley, Esther. «Global Pokénomics gives businesses a real-world boost.» 29 July 2016. The Guardian. 18 September 2017. <https://www.theguardian.com/technology/2016/jul/29/pokemon-go-pokenomics-smartphone-game-shops-cafes-museums>.
Baldwin, Carliss Y og C. Jason Woodard. «The Architecture of Platforms: A Unified View.» September 2008. Harvard Business School. <http://www.hbs.edu/faculty/Publication%20Files/09-034_149607b7-2b95-4316-b4b6-1df66dd34e83.pdf>.
Benner, Katie. «Five Key Moments From Snap’s I.P.O. Roadshow Video.» 17 February 2017. New York Times. 10 September 2017. <https://www.nytimes.com/2017/02/17/technology/snapchat-ipo-video.html>.
Buerli, Mike. «Apple Developers.» 7 June 2017. Introducing ARKit: Augmented Reality for iOS. 14 September 2017. <https://developer.apple.com/videos/play/wwdc2017/602/>.
Chan, Stephanie. «Pokémon Go turns 1 and challenges you to catch all these special events.» 8 June 2017. Venture Beat. 21 September 2017. <https://venturebeat.com/2017/06/08/pokemon-go-turns-1-and-challenges-you-to-catch-all-these-special-events/>.
Confliffe, Jamie. «Amazon Is Reportedly Building Alexa Smart Specs.» 20 September 2017. MIT Technology Review. 21 September 2017. <https://www.technologyreview.com/the-download/608913/amazon-is-reportedly-building-alexa-smart-specs/>.
Constine, Josh. «Apple undercuts Facebook in the augmented reality platform war.» 13 June 2017. TechCrunch. 18 September 2017. <https://techcrunch.com/2017/06/13/augmented-reality-platform-wars/>.
—. «Facebook launches augmented reality Camera Effects developer platform.» 18 April 2017. TechCrunch. 14 September 2017. <https://techcrunch.com/2017/04/18/facebook-camera-effects-platform/>.
Facebook. «Introducing AR Studio.» n.d. Facebook. 21 September 2017. <https://developers.facebook.com/products/camera-effects/ar-studio/>.
Galloway, Scott. «L2Inc / Youtube.» 20 Januar 2017. Scott Galloway at DLD 2017. 14 September 2017. <https://youtu.be/cFxdgZ1az9s?t=7m05s>.
Google. «ARcore.» u.d. Google Developers. 13 September 2017. <https://developers.google.com/ar/>.
Helmond, Anne. «The Platformization of the Web: Making Web Data Platform Ready .» Social Media + Society 30 September 2015.
Isaac, Mike. «this=something I’ve heard from multiple people close to Zuckerberg. Facebook missed out on owning the phone. VR is the next platform to win.» 17 January 2017. Twitter.com. 19 September 2017. <https://twitter.com/MikeIsaac/status/821387038830297088>.
Light, Ben, Jean Burgess og Stefanie Duguay. «The walkthrough method: An approach to the study of apps.» New media & society (2016): 1.
Manjoo, Farhad. «While We Weren’t Looking, Snapchat Revolutionized Social Networks.» 30 November 2016. The New York Times. 21 September 2017. <https://www.nytimes.com/2016/11/30/technology/while-we-werent-looking-snapchat-revolutionized-social-networks.html>.
Mann, Steve. «Mediated Reality with implementations for everyday life.» 6 August 2006. WearTech. 18 September 2017. <http://wearcam.org/presence_connect/>.
Matsuda, Keiichi. «Hyper-Reality By Keiichi Matsuda.» 2016. Hyper-Reality. 19 September 2017. <http://hyper-reality.co/>.
Morozov, Evgeny. To Save Everything, Click Here. NY 10104: PublicAffairs, 2014.
Newton, Casey. «Snapchat adds world lenses to further its push into augmented reality.» 17 April 2017. The Verge. 18 September 2017. <https://www.theverge.com/2017/4/18/15333130/snapchat-world-lenses-something-new-for-facebook-to-copy>.
Nowak, Mike og Guillermo Spiller. «Two Billion People Coming Together on Facebook.» 27 June 2017. Facebook Newsroom. 21 September 2017. <https://newsroom.fb.com/news/2017/06/two-billion-people-coming-together-on-facebook/>.
Parviz, Babak A. «Augmented Reality in a Contact Lens.» 1 September 2009. IEEE Spectrum. 18 September 2017. <https://spectrum.ieee.org/biomedical/bionics/augmented-reality-in-a-contact-lens/0>.
Perez, Sarah. «Pokémon Go creator’s next game will incorporate audio into the AR experience.» 18 September 2017. TechCrunch. 24 September 2017. <https://techcrunch.com/2017/09/18/pokemon-go-creators-next-game-will-incorporate-audio-into-the-ar-experience/>.
Plantin, Jean-Christophe, et al. «Infrastructure studies meet platform studies in the age of Google and Facebook.» 4 August 2016. New Media & Society. SAGE. <http://journals.sagepub.com/doi/10.1177/1461444816661553>.
Schiffer, Alex. «Should augmented-reality games like ‘Pokémon Go’ place limits on the real-world locations they include?» 20 July 2017. Los Angeles Times. 24 September 2017. <http://www.latimes.com/business/la-fi-tn-pokemon-opt-out-20160715-snap-story.html>.
Sicart, Miguel. «Reality has always been augmented: Play and the promises of Pokémon GO.» Mobile Media & Communication 28 November 2016: 30-33.
Slater, Tom. «Introducing ARCore: Augmented Reality at Android Scale (GDD Europe ’17).» 5 September 2017. Youtube. 18 September 2017. <https://youtu.be/rFbcOGuDMPk?t=19s>.
Vanian, Jonathan. «This Is The Next Battleground For Apple, Google, Facebook, and Microsoft.» 7 June 2017. Forbes. 18 September 2017. <http://fortune.com/2017/06/07/apple-google-facebook-microsoft-are-battling-augmented-reality/>.
Wingfield, Nick og Mike Isaac. «Pokémon Go Brings Augmented Reality to a Mass Audience.» 11 July 2016. New York Times. 14 September 2017. <https://www.nytimes.com/2016/07/12/technology/pokemon-go-brings-augmented-reality-to-a-mass-audience.html>.
Zuckerberg, Mark. «Today at our F8 developer conference, we’re making the camera the first augmented reality platform.» 18 April 2017. Facebook. 14 September 2017. <https://www.facebook.com/zuck/posts/10103658596105871>.