The Internet’s Invisible Cleanup Crew

A hidden army of tens of thousands of content moderators is at work every day — in often appalling conditions — to make the internet as we know it habitable. We should hold Silicon Valley responsible.

The business process outsourcing (BPO) industry, which includes commercial content moderators, is one of the fastest growing in the Philippines. (ILO / R. dela Cruz)

At the heart of the Eastwood Mall in Quezon City, Philippines, stands a monument to “the men and women that have found purpose and passion in the business process outsourcing industry.” Ringed by flying steel birds, a man and two women preside over this small patch of Metro Manila, decked out in headsets and briefcases and facing a proud future. Steps away is a Coffee Bean & Tea Leaf where Sarah T. Roberts interviewed a group of commercial content moderators for her book, Behind the Screen: Content Moderation in the Shadow of Social Media. Commercial content moderation, or CCM, describes one of the dirtier jobs on the corporate internet: reviewing and removing violent, racist, and disturbing content posted to social media sites like Facebook and YouTube and in the comments sections of brand-aware websites for consumer products. Such dirty work falls to people like these three cast-iron office workers who do that digital piecework just steps away from the mall.

In Roberts’s telling, the economic and social factors that produce our relatively sanitized experiences online also produce a global class of workers tied to one another by the shared conditions of commercial content moderation: contingency, race-to-the-bottom wages, ever-increasing work speeds and quotas, and exposure to the worst elements of humanity, again and again and again in round-the-clock shifts. Roberts interviews individual workers from the many sectors where content screening gets done: Filipino BPO (business process outsourcing) workers who take CCM jobs when more competitive call-center positions aren’t available; young college graduates in San Francisco whose big tech dreams turn into long-term-temp-worker realities; and American contracting company executives who market their human moderators as American, a xenophobia that feels right at home with contemporary US politics. Some of her informants focus on their own psychological trauma from the content they repeatedly engage. Others lament declining wages, job instability, and the lack of employer benefits as the worst parts of the work. Online, Roberts argues, the pleasure we take in sharing kid photos and memes and online petitions is made possible only by the business-processing workers from Manila to Silicon Valley, all clicking through the dregs of the social web, rendering the space safe from snuff films and child pornography.

Human moderation of online content is not new. In the early days of the web, Roberts recalls from her own experience, moderation was itself a social affair. “Mods” were of and by the communities they moderated, their roles known and respected by participants. Moderation in these contexts was about maintaining and facilitating norms so that the MUD or BBS or Usenet group could function collegially. Everybody knew that moderation was essential to a functional game setting or chat room, and while the early days were certainly not conflict-free, the role and importance of moderators were not in dispute.

Fast forward to the contemporary internet, and content moderation looks somewhat different. No longer the domain of community participants whose identity is known by all, mods are invisible, the fact of moderation, says Roberts, almost secret. Roberts attends to this secrecy in her book, using pseudonyms for informants and some of the companies they work for, mindful of the corporate secrecy that puts these workers at risk. Responsibility for upholding community norms online is carried out by workers in order to enhance the profit of the large-scale social-media companies they work for. The rules of engagement are subject to non-disclosure agreements, moderators work from cubicles and call centers, and the job is no longer a labor of community love, but just another low-paid contingent job from Quezon City to Silicon Valley.

In addition to this small group of Filipino workers, Roberts conducted extensive interviews with a handful of commercial content moderators working in Silicon Valley. That her research takes her to two sites where global capital is on glittering and flamboyant display offers a marked contrast to the working conditions of both. The California content moderators are largely recent college graduates taking jobs in this sector in the hopes that moderating content for Facebook and companies like it will lead to careers in the highly paid technology sector. What they find instead is that they occupy the growing sector of outsourced and contract workers of the new “gig economy.” While they may review content for name-brand companies, their checks come from temp agencies. Their positions are term-limited, pay higher than you make slinging coffee at a Starbucks but vastly lower than you’d get with a “real job” with the company, and lack the fringe benefits like sick time, vacation time, and health insurance that make working in America feasible.

On the other side of the planet, their Filipino compatriots face something much the same: working for contractors, not companies, with wages that once were relatively high in the local labor market, but have been plummeting since the mid-2000s in the face of a global race-to-the-bottom in wages for workers. While American screeners talk a lot about the difficulty of what they see, Roberts’s Filipino workers talk more about the trade-offs of non-voice BPO work: the pay is drastically less, but there’s no dealing with angry Americans screaming at you over the phone.

In Roberts’s analysis, the perils of this line of work are shared across national boundaries. Commercial content moderators face similar working conditions at Megaworld and in the unnamed American corporations in Silicon Valley: primarily contract work and its depressed salaries and lack of benefits; exposure to extreme violations of social norms around sexuality and violence and the attendant psychological harms that stem from that. Roberts focuses on the emotional lives of her informants and the psychological impact of scrubbing the internet clean, particularly in the American context. The discussions Roberts documents from the Philippines sound different. We read less about the need for workplace mental health interventions, and more about how competition with India to provide US corporations with cheap contract labor is increasing the speed and number of images the Quezon City workers must moderate each shift. It is a reminder of the ideological and sociohistorical structure of feeling: what matters to these workers as they click through the worst of the internet depends on the context in which each image flickers past.

What the San Francisco and Metro Manila workers have profoundly in common is both the necessity of their work—Roberts makes a clear case that the internet without commercial moderators would be impossible to navigate—and their utter disposability at the hands of global tech giants. Throughout the book, Roberts refers to commercial content moderators as “professional,” a designation that goes undefined and seems out of step with the labor conditions she describes. “Professional” connotes briefcases and headsets like those on Eastwood Mall’s BPO monument, not the unstable temporary contracts and intensifying quotas that actually structure these positions. The term reads almost like a wish: if these tasks could be professionalized, the workers would be respected, paid well, and given benefits. This is, of course, not how working conditions are transformed, and Roberts offers an array of potential remedies: legally mandated transparency around moderation policies, changes in social media consumption habits on the part of consumers, mental health counseling for moderaters, and labor organizing, which is already taking place among tech workers in San Francisco and Seattle, and among BPO workers in Manila.

The challenge, then, is to imagine ways for workers kept very much behind the screen, separated from one another by miles, by computer interfaces, and by outsourced contracts, to find common cause together against the Facebooks of the world..