Deepfake Atmospheres and Digital Crowds

It has not been a good year for crowds. Around the globe, concert halls are silent; stadia, empty. The decibel level of humanity is very likely at a century low. Throughout the world televised sports—a commercial enterprise for whom atmosphere, ambiance and audiences constitute an essential component of the product itself—broadcasters are turning to technology to navigate the void: spurred by the hope that digital deepfakery (in the form of ‘Atmospheric Audio’) may provide enough of a plug to fill the crowd-shaped hole

II. A Brief History of Digital ‘Sweetening‘

Broadcasting has a long has history of employing tactics of audio interference. In 1945, Jack Dadswell created the first ‘laughing record’ for his WWJB Florida radio station in Florida. A decade later, working at CBS, Charley Douglass brought his own version of the technology, the ‘laugh box,’ to the world of television.

Since then, the ‘sweetening’ of audiences became common practice—for prerecordings and live airings alike. For occasions such as the Academy Awards, producers have long counterbalanced the untimely laughter and unenthusiastic applause of analogue audiences with their digitally ‘programmable’ counterparts. Indeed, for the express purpose of atmospheric deepfakery, live events are almost never ‘live’ at all, but rather televised via a seven-second delay; a buffer which not only allows sound engineers to mute swear words and cut controversial statements, but to ‘autotune’ the atmosphere of the event itself [1].

In the augmented aurality space, however, showbusiness is not alone. Named after Jack Foley, the sound engineer of the silent movie era, foley is the practice of adding everyday sound effects to dramatise the aural landscape of movies, television and, in a slightly more underhanded manner, sports [2]. Indeed, for as long as television has been luring crowds from stadia to sitting rooms, sports broadcasters have admixed foley (oftentimes phony) sounds to keep the ‘liveness’ alive. For instance, when one watches televised horseracing, according to Bill Whiston, former producer of the BBC, horses are nowhere to be heard: “So, basically, the way you cover all of that sort of stuff is to run a tape which has the sound of horses hooves galloping, which is actually, if I remember correctly, a slowed down buffalo charge” [3].

III. Atmospheric Audio

Even pre-Covid, all major sports leagues employed sound engineers to ‘sweeten’ the sound of the already beautiful game. In his article “Meet the man who plays crowd noise sounds on a drum machine for Sky Sports Football,” Ben Allen interviews interviews Adam Peri, a sound engineer, who was already ‘sweetening’ Premier League games long before Covid came around—namely, by following the ball around the pitch via 17 microphones [4].

Since August of this year, however, Sky Sports have have reassigned Peri to an entirely trickier type of sound: crowd noise. Before the Premier League returned, Sky Sports brokered a deal with EA Sports, creators of the FIFA videogame, for with their ‘Atmospheric Audio’ system, a digital archive of about 1,300 sound effects (over 13 hours long) that includes, as per sports journalist Tom Wiggins, “team-specific chants, ‘oohs’ for near misses, and howls of derision for when decisions don’t go the way of the home team” [5]. And thus far, the experiment seems to be (literally) paying off.

When Match of the Day presenter Gary Lineker polled his Twitter followers, the 181,000 responses were resounding, And according to Kevin Ramsay, director of studios at Sky, the results match the data. “Percentage-wise, it has borne out that significantly more people prefer the augmented version than the natural sound” [6].

IV. The Scientific Rationale

For decades, the centrality of crowds to sports has not only been common knowledge, but settled science. In their 1977 analysis of Major League baseball, psychologists Schwartz and Barsky found that home advantage not only increased with crowd density, but that crowd noise had a dramatic effect on the decisions made by referees [7]. (There is a reason why the NFL’s Atlanta Falcons were penalized by the league for pumping artificial crowd noise through their stadium speakers) [8]. In their paper on “Mediated Intra-Audience Effects,” Cummins and Gong cite the work of fellow researchers Lessiter and Freeman, who found that adding base (or merely increasing the volume) enhanced ratings of spatial presence in onscreen environments [9]. “Likewise,” Cummins and Gong continue, “one recent study of video game consumption showed that improving sound quality (i.e., surround sound vs. stereo) enhanced players’ sense of immersion and enjoyment of the game, whereas improving image quality did not yield the same effects” [10]. From the perspective of the ticket holders themselves, researchers Uhrich and Berkenstein have noted that “a noisy crowd and high social density on the stands and bleachers contribute to spectators’ positive emotional experience of the stadium environment” [11]. In close proximity, physical inputs (individuals) produce a metaphysical output (atmospheres); a synthesis of “intra-audience effects,” as John E. Hocking termed them [12], which amounts to more than the sum of its parts. “In a more active role,” Uhrich and Berkenstein continue, “the consumers not only are members of the audience, but also function as co-performers. The typical appearance and behaviour of the fans in the stands are part of the show that are collectively produced and consumed” [13].

V. The Commercial Rationale

In his 1975 study of the Super Bowl, Michael R. Real observed that only 3% of the televised event featured actual athletic competition [14]. Two years later, building on this observation, B. R. Williams noted a growing divide between ‘the game event,’ the actual on-field competition, and ‘the stadium event,’ which Williams describes as “the total sequence of activities occurring in the stadium, both perceived and participated in by the fans” [15]. In this total sequence of activities, aurality is a primary channel for broadcasters of televised sports broadcasters to raise their information rate. “The more stimuli exist, and the stronger they are” explain Uhrich and Koenigstorfer, “the higher—and thus more emotionally rousing—the information rate” [16]. Indeed, even placing aesthetics to one side, the augmentation of aurality makes commercial sense; for, as Uhrich and Koenigstorfer have shown, “environment-elicited emotional states have been shown to influence shopping-related behavioural reactions, such as the length of time spent in the store and unplanned increases in spending” [17]. For shopping outlets and sports broadcasters alike, atmospherics create a kind of positive feedback loop. The bigger the the crowd, the bigger the crowd noise; the bigger the crowd noise, the bigger the crowd.

VI. Conclusion

In “C’mon Feel the Noise: Football Fans and Artificial Crowd Noise,” Gary Sinclair writes of how “silence in stadiums has reminded big business and audiences of the centrality of fans to sport” [18]. Yet, as Sinclair also points out, match-day income had already plummeted (from 43% in 1992 to 13% in 2018) long before Covid-19 came around. Today, the market for sociality, solidarity—the ‘stadium experience’ writ large—has never been greater. The question remains, however—as technology improves, audiences acclimatise and overheads fall—will ‘live’ audiences become a thing of the past? Throughout the corporate world, companies are breaking leases and furnishing the ‘home offices’ of employees. As working from home becomes one of the many ‘new normals,’ one wonders if watching from home is heading down the selfsame path.

References:

[1] Keishin, Jennifer. “Where does canned laughter come from—and where did it go?” BBC, 26 September 2016. https://www.bbc.com/culture/article/20160926-where-does-canned-laughter-come-from-and-where-did-it-go.

[2] Park, William. “The Secret Sounds in Movies and TV,” BBC, 16 June 2016. https://www.bbc.com/future/article/20150616-the-secret-sounds-in-movies-and-tv.

[3] Madrigal, Alexis C. “Everything Is a Remix: The Sound of Horses Racing on TV Is Actually a Sample of Buffaloes Charging,” The Atlantic, 18 July 2012. https://www.theatlantic.com/technology/archive/2012/07/everything-is-a-remix-the-sound-of-horses-racing-on-tv-is-actually-a-sample-of-buffaloes-charging/260001/.

[4] Allen, Ben. “Meet the man who plays crowd noise sounds on a drum machine for Sky Sports Football,” GQ, 10 July 2020. https://www.gq-magazine.co.uk/sport/article/artificial-crowd-noises.

[5] Wiggins, Tom. “Football’s Canned Atmosphere: How Crowd Noise Is Added to Empty Premier League Grounds,” Techradar, 19 June 2020. https://www.techradar.com/news/footballs-canned-atmosphere-how-crowd-noise-is-added-to-empty-premier-league-grounds.

[6] Allen, ibid.

[7] Nevil, A. M., N. J. Balmer & A. Mark Williams, “The Influence of Crowd Noise and Experience Upon Refereeing Decisions in Football,” Psychology of Sport and Exercise, Vol. 3 (2002).

[8] Cummins, R. Glenn & Zijian Gong, “Mediated Intra-Audience Effects in the Appreciation of Broadcast Sports,” Communication & Sport, Vol. 5, No. 1 (2017).

[9] Ibid.

[10] Ibid.

[11] Uhrich, Sebastian & Martin Berkenstein, “Sport Stadium Atmosphere: Formative and Reflective Indicators for Operationalizing the Construct,” Journal of Sport Management, Vol. 24 (2010).

[12] Cummins & Gong, ibid.

[13] Uhrich, Sebastian & Martin Berkenstein, “Physical and Social Atmospheric Effects in Hedonic Service Consumption: Customers’ Roles at Sporting Events,” The Service Industries Journal, Vol. 32, No. 11 (2012).

[14] Cummins & Gong, ibid.

[15] Ibid.

[16] Uhrich, Sebastian & Joerg Koenigstorfer, “Effects of Atmosphere at Major Sports Events: A Perspective from Environmental Psychology,” International Journal of Sports Marketing and Sponsorship, Vol. 10, No. 4 (2009).

[17] Ibid.

[18] Sinclair, Gary. “C’mon Feel the Noise: Football Fans and Artificial Crowd Noise,” RTE, 9 July 2020. https://www.rte.ie/brainstorm/2020/0709/1152260-football-soccer-sport-artificial-crowd-noise/.