Recognition: looking through a mediated lens

“Can a machine make us look afresh at great art through the lens of today’s world?” (Tate). This is a question art institution Tate Britain poses about their online program Recognition, launched two weeks ago. This program uses Artificial Intelligence (AI) software that makes connections between British artefacts (mostly paintings) from the Tate collection and contemporary press photos from Reuters. The program won the IK price 2016, a contest supported by Microsoft (Davis). Recognition is available on a web site where users can find the best results from the database. There is also an installation in Tate Modern, where people can interact with the program. This blog will discuss how Recognition works, what makes it new and what the limitations of the program are.

How does it work?
The AI software compares a database of images by using object and facial recognition and composition analysis. It can therefore indicate similarities in press photos and art works by looking at compositions, faces, gender, age and emotional states, objects and by reading metadata such as titles, tags, dates and descriptions (Recognition; Dunne). The picture below shows the interface of the Recognition website. On the left is a press photo of the Dalai Lama from 16 September 2016, on the right a painting by Sir John Lavery from 1926. The text describes what similarities has been detected, to what percentage the compositions and contexts match and what the similar tags connected to the pictures are. Interesting fact: the software did not detect the faces in both the images.

Meaningful artware
So, why did Recognition win the IK prize? On its website Tate states that the program is relevant “for the purpose of stimulating debate about art, expression and representation”(Tate). The makers of Recognition, emphasize the possibilities of technology in general: “Technology is an empowering tool (..) You can create really meaningful things and really helpful things, and beautiful things with technology”(Davis). What makes Recognition new, is that it takes art works out of their regular context and into a new one by matching them with contemporary press photos. This places both types of media in a new perspective. The AI software can therefore be described with the term ‘artware’ which derives from art and software. Artware could function as: “proposals for the restructuring or critique of existing media systems”(Paul 16).

But one could argue whether Recognition can really restructure media systems or stimulate debate. Natalia Grincheva sees opportunities for museums acting as a “trans cultural encounter” (45). But she doubts if a cultural institution, as Tate, “could really play a vital role in facilitating cross-cultural understanding” (46). It is uncertain if Recognition will be useful outside the context of the museum. This is something that is not quite clear yet, because the program just started. In addition, it needs to be taken into account that the software is not neutral and “has to be seen as cultural construct”(Paul 16).

Another question is whether AI can truly understand an image (Davis). Because Recognition focuses on similarities in images and uses four tools to find these, some characteristics of both forms of media will not be taken into account. The four elements the software recognizes and compares give a simplified view of what an image really contains or means. This is supported by the authors Chae et al., who compared how curators made categories and how software did this. They conclude that “automatic clustering cannot replace human judgment completely” (898). Recognition does not look the same way at the images as humans might do. By letting visitors of the Tate give feedback to the AI, the matches Recognition makes can be improved. So, the addition of human feedback is an important part of the program.

The mediated lens
What becomes clear is that Recognition places the Tate collection in a new, contemporary perspective. The question is how meaningful this form of artware can be. While this may be a question that can only be answered in the future, Recognition already shows the possibilities of AI for art and museums. But it is important to understand how the technology mediates the perception of images and the contextualization of it. The question that Tate poses, as mentioned in the introduction, might be answered with the fact that this new lens’ can indeed broaden our view of images and their contexts, but does so through a mediated lens, which is important to acknowledge.

Bibliography
Chae et al. ” Linking and Clustering Artworks Using Social Tags: Revitalizing Crowd-Sourced Information on Cultural Collections.” Journal of the Association for Information Science and Technology 67.4 (2016): 885-899.

Davis, Nicola. “Tate Britain Project Uses AI to Pair Contemporary Photos with Paintings.” The Guardian. 28 Aug. 2016. 18 September 2016. <https://www.theguardian.com/artanddesign/2016/aug/28/tate-britain-project-recognition-artificial-intelligence-photography-paintings>.

Dunne, Carey. “Tate Uses AI to Match Old British Art to New Photojournalism.” Hyperallergic. 9 Sept. 2016. 17 September 2016. < http://hyperallergic.com/321407/tate-uses-ai-to-match-old-british-art-to-new-photojournalism/>.

Grincheva, Natalia. “Cultural Diplomacy 2.0: Challenges and Opportunities in Museum
International Practices.” Museum & Society 11.1 (March 2013): 39-49.

Paul, Christiane. “Digital Art/Public Art: Governance and Agency in the Networked Commons.” First Monday 7 (2006). 17 September 2016. <http://firstmonday.org/ojs/index.php/fm/article/view/1616>.

Recognition. 2016. 18 September 2016. < http://recognition.tate.org.uk/>.

Tate. “IK Prize 2016: Recognition.” 30 August 2016. 16 September 2016. <http://www.tate.org.uk/whats-on/tate-britain/exhibition/recognition>.