Join the HIVE: The AR future of social networks

The Age of AR

Augmented Reality (AR) is set to be the next big step in computing,^[1] [2] and experts are predicting a big growth of the AR sector, estimating it at around 140 billion Euros worth by 2024. ^[3] Many of the technology “giants” are working on their own augmented reality projects, each with their own take on it. Google has developed Google Lens, an augmented reality service allowing users to enhance their surrounding by adding metadata; Facebook on the other hand is developing their own augmented reality platform with a very distinguished facial recognition technology. Lastly, Microsoft’s Hololens offers a quick gaze in the nearby future, with a device capable of showing augmented reality in real life in three-dimensional surroundings. However, none of these companies have been taking concrete steps towards a comprehensive AR social environment uniting these three technologies.

And yet, this is arguably the likely upcoming stage of this technology in the foreseeable future. Our prognosis is that AR’s next step will include technology that is able to recognize human faces in real time, as part of an immersive comprehensive AR-based social network. To prove this point, we will initially provide an analysis of the technological state of the art by looking at three big companies that are already heavily investing in AR: Google, Microsoft and Facebook. Based on expert opinions and prognoses, we will go on to predict this near future of AR technology and visualize it by presenting a mock-up of an application that lies at the end of those current trends, which we call Hive. Finally, we will discuss the possible implication of such an app becoming reality – for better or for worse.

Google Lens and Machine Learning

Google Lens employs a software that allows users to enhance the interaction with their environment using their camera. Essentially, Google Lens is able to recognize most items the user points the camera at. To recognize, categorize and name pictures, the software uses machine learning: the software learns from examples and experience, rather than a given set of fixed rules. These examples and experiences then form a dataset, teaching the software to search for certain patterns and to recognize deviations from these patterns, which further aids in the categorization of objects.^[4]

This enables Google Lens to recognize an object in a picture by comparing it with other pictures, looking for similarities and finally using the metadata of the other pictures to name the object. Since the most recent update, Google Lens is capable of a wide array of things, including:

• • Understanding a text
  • Finding similar products to an chosen object
  • Identifying plants and animals
  • Recognizing books and media
  • Scanning barcodes^[5]

 

By learning from past user input and experience, machine learning helps improve and personalize search results and employs its very usage to refine its working – even turning accidents into assets, in the words of Benjamin Bratton.^[6]

But while trends like the Google Lens have set the precedent for AR technology made available to a broader audience, this is merely the beginning. With a “Machine-Learning Revolution” already being hailed in 2018,^[7] this AI feature is changing the way we interact with software and algorithms – and with each other through them. Machine learning will likely be used as a central tool in marketing and advertising and, in the long run, deeply impact social networking, as well.^[8]

Facebook and Facial Recognition

Where Google Lens ends, Facebook’s AR technology begins. While Lens is able to recognize objects, animals and plants, Facebook has placed its focus on recognizing the human face. This facial recognition technology is used by Facebook for:

• • Detecting if a user is pretending to be someone they’re not.
  • Giving suggestions for tagging people in pictures.
  • Letting users know when they appear in a picture/video, but are not tagged.
  • Providing alt-text(information about pictures on the website) for the visually impaired^[9]

 

Facebook’s facial recognition technology works similar to that of Google Lens, analyzing profile pictures and tagged photos and videos to search for facial patterns. Facebook calls this “making a template”.By continuously comparing this template with other pictures and videos, Facebook is able to recognize a human face. The more pictures are being uploaded to Facebook, the more thorough and accurate the facial recognition becomes.^[9]

Facial Recognition in itself is expected to become a central aspect of future social networking as well.^[10] While in many early trials, the technology could easily be fooled (i.e. by a two-dimensional picture of a person), the trend is heading increasingly towards a three-dimensional full body recognition of users. Early examples of this include the iPhone’s facial unlocking and its Animojis; but as Acquisti points out in his TED talk, this is only the tip of the iceberg when it comes to possible applications.^[11]

From a pay-by-face authentication system being trialed in Finland to personalized ads that feature pictures of your trusted friends,^[11] facial recognition allows content, and especially advertising, to be increasingly personalized and focused – thanks to highly advanced real-time VR that recognizes, uses and provides information to us based on our surroundings, past actions and preferences. Biometric sensors and iris scanning are only two of the seemingly endless applications of facial recognition which, according to experts, will ultimately lead to facial recognition becoming so ingrained in everyday life that it will be hard to avoid.^[12]

Microsoft Hololens and Holographics

Microsoft is approaching the AR sector with its Hololens; likely the starting point for a large industry sector. Hololens is the first device capable 3D holographic images in real time in the real world – which means that users are able to walk around digital objects, interact with them and share them with other users, all in real time.^[13] This is what fundamentally differentiates the Hololens from other AR devices and services: the fact that rather than superimposing information on the field of view through the screen, objects are made to look like three-dimensional objects placed within the physical environment.

Hololens achieves this with five environment scanning cameras, a depth perspective camera, a motherboard with GPU CPU and – newly – HPU (holographic processing unit), WIFI and bluetooth connection and spatial speakers that are capable of tricking the user into thinking the sound comes from a certain point in the room.^[14] All of these technologies work together to create the illusion of a digital moving and interactive object in the real world.

(De)Centralization of Social Networks

The connection of AR technology with actual social media platforms is imminent: In fact, Facebook CEO Mark Zuckerberg predicts VR systems as potentially “the most social platform”.^[10] However, the incorporation of new technologies isn’t the only direction in which the big media giants like Google, Microsoft and Facebook are expanding. With Facebook trialing dating platform features and having acquired competitors like Instagram and WhatsApp, platforms appear to be heading towards a clustering of services within small number of monopolies that are merging the private, the public and the professional sphere.

This trend, however, is unlikely to continue endlessly. The monopolization of services by Facebook, Google and Amazon are already encountering harsh criticism and claims of unlawful domination of the market,^[15] with many calling for the “giants” to be broken up into several sub-services and their acquisitions to be more strictly controlled, comparable to what happened to Microsoft in the early 2000s.^[16] A such break-up would open the market to smaller competitors with an offer of “glocal” services; with a less universal scope than Facebook and focused on smaller networks, which could be based on concrete goals and objectives or even sub-communities, such as families or neighborhood connections.^[17]

Visualizing the next step

It is at this moment that we are locating our research project: Based on the current trends in social media organization and technology and the prognoses on how they will continue, we have created a mock-up of a social media application, called Hive.

Based on the referenced predictions, the hypothetical launch of Hive would take place between 2020 and 2024: this is the timeframe most current prognoses refer to. If the break-up of the big tech monopolies, as envisioned by some experts quoted above, will in fact become reality, this would open the market for smaller social media competitors and enable their success. In this case, a platform like Hive would not be unlikely to be introduced and adopted.

On a technological level, Hive incorporates the three central technological aspects mentioned above: machine learning, facial recognition and a real-time aspect. Regarding content, we have envisioned Hive as a glocal platform, using location services and the recognition of surroundings for targeted advertising – conjoining aspects from social networks and professional networks. Social networking in Hive is a fundamental part of the user’s world view, since interactions with people and objects in the field of vision are mediated through the AR aspects, such as additional information or suggestions.

The future AR social medium

According to Evgeny Morozov, Augmented reality is ‘the fancy term for infusing our everyday environment with smart technologies’.^[18] This is precisely what Hive aims to do. This application immerses the user fully into the digital version of the world with potentially unbridled potential.

Figure 1 depicts the login screen for the application. Here the front camera can be used to allow login on any device by using a patented facial recognition system. One of the most useful properties of Hive is that it synchronizes your data from a range of online platforms including Google calendars and your interests from Facebook.

Figure 2 shows Hive in action. Here, the rear-facing camera recognises the environment and points the way to two vending machines with information about content and payment method. The application also detects two faces of fellow Hive users: one friend and one ‘potential friend’. Hive takes your contacts and events from other platforms and instantly matches them to anyone that the user might come across on the street, whether that person is already connected with the user or not. These people can then be suggested as ‘potential friends’: people that have similar interests, events or friends in common. Upon meeting a ‘potential friend’, the user will be presented with a selection of their latest social media posts in order to provide talking points, or emphasise similarity. The tool bar at the top is intentionally integrated to the interface in order to encourage immersion.

Finally, ‘Figure 3’ depicts the navigational and informative aspect of the application. When pointed at landmarks or points of interest the application offers what Mark Graham calls Google’s ‘virtual palimpsest of place’.^[19] This presents upcoming events, booking details and information at the push of a button. The application uses geotagged information and machine learning to enhance the experience of other users. As is the case for most Internet based platforms, the main source of income is advertisements. Figure 3 depicts one such type of advertisement that is based on user history, location and even time of the day to deliver advertisements to the user’s field of vision in real time. Additionally, when users joins the Hive, they will never be lost (as long as they have battery on their device). This is because the application superimposes information from Google Maps allowing the user to seek specific directions, or share their location with friends.

While the current technology is tethered to handheld devices, there is hope that Augmented Reality will truly become ubiquitous with the introduction of hands-free hardware such as in the case of Google Lens.

Privacy at stake

While advantageous in many ways, Hive also entails aspects that are open to critique, particularly issues of privacy and surveillance. A primary privacy concern is the gathering and further use of user data. According to Kitchin, we live in an era of “widespread data generation and surveillance”, which raises questions  about the extent of privacy and human rights infringement through “data generation and […] dataveillance.”^[20] Broader options for such dataveillance open the door for the use and abuse of user data, e.g. the selling of data to corporations for marketing purposes – thereby allowing companies to directly generate money through user data.

Through the geo-locational technologies and real-time aspect in our application – as well as the general ubiquity of AR technology in everyday life – user location is continuously tracked and  processed, including the current location at any given time, past locations and frequently visited sites or routes.^[20] This almost universal locational data could potentially be of high interest for hackers; how it is used and protected within the technology is therefore important to know.

The second issue relates to the privacy of people choosing to opt out of using this technology. With facial-recognition software, they are still being recorded and scanned, presumably without their knowledge and consent. This raises the question of to what extent users are able to opt out of this technique, posing an ethical dilemma. Without being an active registered user yourself, you could become a ‘ghost user’ that is already on the server due to data provided by your friends and contacts.^[21] This essentially means that there is information about non-users already stored in the database, such as visited places – and this information is waiting for the users to activate their respective profiles so this data can be actually linked to them.

A real-life filter bubble

Our application, embedded in everyday life, could also have social implications and lead to what could be called a real-life filter bubble,a derivation of the term ‘filter bubble’ coined by Eli Pariser. Pariser describes the filter bubble as the “unique universe of information” that is created for each of us, fundamentally altering the way we encounter ideas and information”.^[22] A real-life filter bubble could therefore contribute to social division, due to it facilitating and promoting only superficial interaction with people who at first glance share the same characteristics or interests – to the detriment of chance encounters and the diversity of social contacts. An individual’s personality can hardly be comprehensively understood or evaluated based on basic facts about them – our application therefore has the potential to further social clustering based on superficial compatibility.

Safety issues

Numerous articles written about Pokémon GO since its release have issued safety concerns when it comes to using the app.^[23],^[24] There have been reports of people’s lowered attention leading to traffic accidents, due of the immersive qualities of the app. Similar cases have been reported about Google Glass users having limited vision or simply not paying attention – distracted by the visual information provided to them.^[25] The AR application we propose shares central features with these two examples – similar safety concerns therefore apply.

On an ethical level, another safety concern arises from the kind of personal information that is to some degree general publicized through AR and facial recognition. Intimate data such as sexual orientation, if liked to the profile, would be either easily visible or at least prone to being hacked and exploited. Especially in a time of discrimination and marginalization of minorities, the reveal of people’s intimate information would make them targets for advertising, but also for abuse.

Conclusion

Hive – a social media platform including Augmented Reality features. A thing of the future still, but according to tech experts, the not too distant future. With AR technology set to become the next big step in computing, it is merely a question of time until Hive – or a similar platform – becomes reality and changes the way we interact, on-screen and off-screen. And in times of AR, that very distinction will become increasingly blurred.

So while we still find ourselves in the first step of this trend, it’s the right time to ask where this development might be going, and how it will impact our everyday lives. While an application like Hive, machine learning superimposes the vast database of a search engine on its users field of view, and combined with facial recognition, makes it virtually impossible to opt out: People are going to be recognized, users or not. With hands-free usage and personalization as perks, but privacy and safety issues as possible dangers of this next step, our analysis yields a final answer about the future of social networking: it isn’t good or bad, it’s inevitable. So before it arrives, it is important to face its implications – for better or for worse. And this research project might be the first step.

References

 

^[1] AR in ACTION. “Aaron Luber | ARIA”. Online video clip. YouTube,Youtube, 28. Mar, 2018. Web. 23 Sept, 2018.

^[2] Tynan, Dan. “The Future Will Be Augmented. Why Facebook, Google, Snap, and dozens more love AR – and what’s coming next” Fast Company South Africa. Oct 16, 2017. Fast Company. 12 Oct, 2018.

^[3] “What future for Augmented Reality?”. BBC News, BBC. 02 Oct, 2017. Web. 14 Oct, 2018.

^[4] Google Cloud Platform. “Intro to Machine Learning on Google Cloud Platform (Google I/O ’18).” Online video clip. YouTube, YouTube, 10 May, 2018. Web. 25 Sept, 2018.

^[5] Brown, C. Scott. “Google overhauling image search, bringing Lens to Google Images”. Android Authority. Android Authority, 24 Sept, 2018. Web. 11 Oct, 2018.

^[6] Bratton, Benjamin H. The Stack:. On Software and Sovereignty. Cambridge: MIT Press, 2013. Print.

^[7] Evans, Michael. “The Machine Learning Revolution: How Artificial Intelligence Could Transform Your Business”. Forbes Media LLC, 20 Oct, 2018. Web. 21 Oct, 2018.

^[8] Darling, Thomas. “Big AR: The Current State of Augmented Reality”. Medium. Amazon Registrar, Inc., 27 Jun, 2017. Web. 14 Oct, 2018.

^[9] “How Does Facebook Suggest Tags?”. Facebook Help Center, Facebook, 2018. Web. 19 Oct, 2018.

^[10] Robertson, Adi. “Mark Zuckerberg says augmented reality glasses are ‘what we’re trying to get to’. The Verge. Vox Media Inc., 12 Apr, 2016. Web. 17 Oct, 2018.

^[11] Torgovnick May, Kate. “The future of facial recognition: 7 fascinating facts”. TED Conferences, LLC., 17 Oct, 2013. Web. 15 Oct, 2018.

^[12] Dans, Enrique. “Facial Recognition And Future Scenarios”. Forbes Media LLC, 25 Jun, 2018. Web. 18 Oct, 2018.

^[13] Carson, Erin. „What HoloLens means for Microsoft and for the future of augmented reality”. CBS Interactive, Apr 22, 2016. Web. 12 Oct, 2018.

^[14] TheVerge. “Microsoft HoloLens Teardown”. Online video clip. YouTube, YouTube, 6 Apr. 2016. Web. 03 Oct, 2018.

^[15] Mims, Christopher. “Tech’s Titans Tiptoe Toward Monopoly”. The Wall Street Journal. Dow Jones & Company, Inc., May 31, 2018. Web. 12 Oct, 2018.

^[16] Brandom, Russell. “The monopoly-busting case against Google, Amazon, Uber, and Facebook. What tech companies have to fear from antitrust law”. The Verge. Vox Media Inc., Sep 5, 2018. Web. 13 Oct, 2018.

^[17] Kastanis, Diomedes. “What Technology Will Look Like In Five Years”. Oath Tech Network. Nov 15, 2015. Web. 15 Oct, 2018.

^[18] Morozov, Evgeny To Save Everything, Click Here: The Folly of Technological Solutionism. New York: PublicAffairs, 2013. Print.

^[19] Graham, Mark. “Neogeography and the palimpsests of place. Web 2.0 and the construction of a virtual earth.”Tijdschrift voor Economische en Sociale Geografie, 101.04 (2013): 422–36. Print.

^[20] Kitchin, Rob. The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. Thousand Oaks, CA: SAGE Publications, 2014. Print.

^[21] Bucher, Taina. ​“The Friendship Assemblage: Investigating Programmed Sociality on Facebook”. Television & New Media06 (2013): 479-93. Print. 13 Oct. 2018.

^[22] Pariser, Eli. The Filter Bubble: What the Internet is Hiding from You. London: Penguin, 2011. Print.

^[23] Barbieri, Stefania, et al. “Pedestrian Inattention Blindness While Playing Pokémon Go as an Emerging Health-Risk Behavior: A Case Report.” Journal of Medical Internet Research4 (2017). Web. 16 Oct. 2018.

^[24] Colley, Ashley, et al. “The Geography of Pokémon GO: Beneficial and Problematic Effects on Places and Movement.” Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. (2017). Web. 16 Oct. 2018

^[25] Knapton, Sarah. “Google Glass could be dangerous for drivers and pedestrians, study warns”. The Telegraph 04 Nov. 2014: 1. Web. 13 Oct. 2018