When Fiction Becomes Reality: From Black Mirror to China’s Social Credit Score
Since 2014, the Chinese government is working on implementing a new behavioral scoring system: the social credit score. Presented as an assessment of the ‘reliability’ of citizens, it combines a whole set of criteria such as consumption habits, records and even friends. Each citizen will be assigned a score, between 350 and 950 points, according to how they behave. From 2020 onwards, the system should be applied to the entire country, and all 1.3 billion citizens will be graded and ranked whether they like it or not. The Next Web reports that, in order to promote societal harmony and build a culture of ‘sincerity’ through this new system, the state considers the internet in China as part of its sovereign territory, and all Internet-based data are therefore exploitable by the government. The Chinese e-commerce giant, Alibaba already has a similar system in place, which allows buyers and sellers to grade each other to improve reliability, but such a system has never been applied to a whole country.
© Kevin Hong
How does it work?
The social credit score uses both big data analysis technologies as well as monitoring techniques such as facial recognition to ‘score’ citizens. Zahy Ramadan explains that every action taken by Chinese citizens will either raise their grades or lower it. Their habits, financial records, behaviors while driving their car, at work or on social media — all of this data will be scrupulously analyzed and will result in a score. For instance, criticizing the government on social media, or mistreating children will lead to a lower grade. On the other hand, driving safely or volunteering for charity will result in a higher score. “A society of control,” is what Léa Polverini, journalist at Slate, calls this emergent system, where citizens will be rewarded or sanctioned based on their score. ‘Model’ citizens can benefit from things like loan approval, getting fast-tracked visa applications, or not having to pay a deposit for rental properties. Those with lower scores encounter penalties and restrictions. They may lose their jobs or the right to travel. The overriding principle is simple: “If trust is broken in one place, restrictions are imposed everywhere,” (Chinese State Council General Office). According to The Verge, this “growing patchwork of social engineering” has already imposed punishments on more than seven million citizens.
A technology with easy drifts
In Persuasive Technology, B.J. Fogg claims that we have entered an era of persuasive technology, of interactive computing systems designed to change people’s attitudes. In this case, a systematic exploitation of big data, invasion of private life, and algorithmic authoritarianism, means that the technology not only incentivizes behavior — it shapes it. Behavioral scoring will provide an instrument of self-censorship. It ‘nudges’ citizens away from purchases and behaviors the government does not like, “The potential for using (or, unfortunately, abusing) such technology is enormous,” (Fogg, 4). Basic citizen freedoms, already undermined by the authoritarian regime of Xi Jinping, will take a new blow. Not to mention the possible leaks and potential for attack, “these gigantic databases offer a treat to companies, but also malicious hackers,” says Marine Jeannin, journalist at Le Nouveau Magazine Littéraire.
Ethical or not, it may be exportable
In Western culture, this kind of ranking system where citizens would be rated based on their personal and professional data would be unimaginable, primarily for privacy reasons. “Chinese authorities clearly hope to create a reality in which bureaucratic pettiness could significantly limit people’s rights,” writes Maya Wang, a researcher for NGO Human Rights Watch. Indeed, such a system raises a lot of ethical questions about the right to privacy, freedom of speech and the role of state power. Should the government be able to “spy” on its citizens at all times? Is it the government’s place to say who is a good person and who is not? Does crossing the road when the light is red make you a bad citizen? Should a black list of “bad” citizens exist and be revealed publicly? Are there any other ways to improve behavior? This system spurs so many controversial debates that the questions are endless.
But if we think about it, rating is already a common process: “We already rate restaurants, movies, books and even doctors. […] We already live in a world of predictive algorithms,” claims Rachel Botsman, author of Who can you trust? How technology brought us together and why it might drive us apart. Wired gives a concrete example when telling its readers to think about their Uber experiences and asking them whether they tend to be nicer with the driver because they know they are going to be rated (Bostman, “Big data meets Big Brother as China moves to rate its citizens”).
In addition, an OECD study made in 2015, revealed that there are at least 24.9 connected devices per 100 inhabitants in the US. Many companies use the “big data” emitted from these devices to understand our lives and desires, predict our actions and target us with advertisements. The social credit score may sound like a crazy project to many, but unfortunately it may be not that far from spreading worldwide, at least that’s where we are heading at according to Wired. In fact, China is already making deals with countries such as Thailand, Myanmar, and Singapore to sell their new monitoring technologies.
Gamified obedience
To enforce stability, the scoring system attempts to make obedience feel like gaming. Their system reflects a cunning paradigm shift because it is a method of social control dressed up as a reward system. Rogier Creemers explains that such a system will create nudges: earning points may influence and ‘nudge’ citizens to do things they may not have done under other circumstances. According to Wired, this system is “basically a big data gamified version of the communist party’s surveillance methods: the disquieting dang’an,” (Bostman, “Big data meets Big Brother as China moves to rate its citizens”). Some people sees this as a positive motivation, others don’t. In my opinion, it is not up to the government to decide who is trustworthy and who is not, especially through such intrusive methods. Many additional ethical problems arise; one risk highlighted by Wired is that life may become an endless popularity contest, with everyone vying for the highest rating that only a few can attain. Another potential problem might be that significantly more powerful citizens might not have their bad behavior affect their score, “As Xi Jinping’s power grows, and as the system approaches full implementation, more abuses will come,” says Maya Wang who qualifies the system as a “dystopian future” (“China’s Chilling ‘Social Credit’ Blacklist”).
© Kevin Hong
Thanks to big data, this new device will provide the Chinese government with an unprecedented tool of repression and censorship. The system will “allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step” (Chinese government, according to The Wall Street Journal). Even though it is still too early to know how a culture of constant monitoring and rating will turn out, it is obvious that, from now on, the important questions about the future are not technological or economic, they are ethical. Welcome to Nosedive (S03E01 Black Mirror): Today, China. Tomorrow, a place near you?
References
Fogg, B.J. Persuasive Technology. Morgan Kaufmann Publishers, 2003.
Creemers, Rogier. “Le pivot de la cybergouvernance chinoise.” Perspectives chinoises, Centre d’étude Français sur la Chine Contemporaine, 15 Dec. 2015.
Ramadan, Zahy. “The gamification of trust: the case of China’s “social credit”.” Emerald Insight, Emerald Publishing Limited, 5 Oct. 2017.
Sataline, Suzanne. “Expert Views: Q&A: Nicholas Loubere on China’s Social Credit System.” Sage business researcher, 30 Apr. 2018.
Van Doorn, Niels. “The neoliberal subject of value.” Cultural Politics, Duke University Press, 2014.
Polverini, Léa. “En Chine, les « mauvais citoyens » ne pourront plus prendre l’avion ou le train.” Slate, http://www.slate.fr/story/159193/en-chine-les-mauvais-citoyens-ne-pourront-plus-prendre-lavion-ou-le-train, 19 Mar. 2018.
O’kane, Sean. “China will ban people with poor ‘social credit’ from planes and trains.”The Verge, https://www.theverge.com/2018/3/16/17130366/china-social-credit-travel-plane-train-tickets, 16 Mar. 2018.
Botsman, Rachel. “Big data meets Big Brother as China moves to rate its citizens.” Wired, https://www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion, 21 Oct. 2017.
Wang, Maya. “China’s Chilling ‘Social Credit’ Blacklist.” Human Rights Watch, https://www.hrw.org/news/2017/12/12/chinas-chilling-social-credit-blacklist, 12 Dec. 2017.
Benaim, Daniel & Russon Gilman, Hollie. “Surveillance de masse: les tyrans du passé en rêvaient, la Chine l’a fait.” Slate, http://www.slate.fr/story/165776/attention-autoritarisme-algorithmique-chine-technologie-surveillance-autoritarisme-dictature-exportation, 11 Aug. 2018.
Jeanin, Marine. “En Chine, un système de « crédit social » réglera la vie des citoyens.” Le Nouveau Magazine Littéraire, https://www.nouveau-magazine-litteraire.com/idees/en-chine-un-systeme-de-credit-social-reglera-la-vie-des-citoyens, 2 May. 2018.
Goodwins, Rupert. “Big data, big results: The IT buzzword that, we must grudgingly admit, actually works.” Arstechnica, https://arstechnica.com/information-technology/2015/10/big-data-an-it-buzzword-that-is-actually-producing-results/, 17 Oct. 2017.
The Conversation. “China’s censored Google is an ethical catastrophe.” The Next Web, https://thenextweb.com/syndication/2018/08/06/chinas-censored-google-is-an-ethical-catastrophe/, 2018.
“Le projet fou chinois : noter les citoyens quoiqu’ils fassent.”Youtube. 3 Nov. 2016. https://www.youtube.com/watch?v=7TMdpsGjoGw