"The dark side of Facebook"

1. TECHNOLOGY»
2. FACEBOOK

Our social networking pages are being policed by outsourced, unvetted moderators.

The invisible army policing Facebook has access to our most intimate secrets

For most of us, our experience on Facebook is a benign – even banal – one. A status update about a colleague’s commute. A “friend” request from someone we haven’t seen for years (and hoped to avoid for several more). A picture of another friend’s baby, barely distinguishable from the dozen posted the day before.

Some four billion pieces of content are shared every day by 845 million users. And while

most are harmless, it has recently come to light that the site is brimming with paedophilia, pornography, racism and violence – all moderated by outsourced, poorly vetted workers in third world countries paid just $1 an hour.

In addition to the questionable morality of a company that is about to create 1,000 millionaires when it floats paying such paltry sums, there are significant privacy concerns for the rest of us. Although this invisible army of moderators receive basic training, they work from home, do not appear to undergo criminal checks, and have worrying access to users’ personal details. In a week in which there has been an outcry over Google’s privacy policies, can we expect a wider backlash over the extent to which we trust companies with our intimate information?

Last month, 21-year-old Amine Derkaoui gave an interview to Gawker, an American media outlet. Derkaoui had spent three weeks working in Morocco for oDesk, one of the outsourcing companies used by Facebook. His job, for which he claimed he was paid around $1 an hour, involved moderating photos and posts flagged as unsuitable by other users.

“It must be the worst salary paid by Facebook,” he told The Daily Telegraph this week. “And the job itself was very upsetting – no one likes to see a human cut into pieces every day.”

Derkaoui is not exaggerating. An articulate man, he described images of animal abuse, butchered bodies and videos of fights. Other moderators, mainly young, well-educated people working in Asia, Africa and Central America, have similar stories. “Paedophilia, necrophilia, beheadings, suicides, etc,” said one. “I left [because] I value my sanity.” Another compared it to working in a sewer. “All the ---- of the world flows towards you and you have to clean it up,” he said.

Who, one wonders, apart from the desperate, the unstable and the unsavoury, would be attracted to doing such an awful job in the first place?

Of course, not all of the unsuitable material on the site is so graphic. Facebook operates a fascinatingly strict set of guidelines determining what should be deleted. Pictures of naked private parts, drugs (apart from marijuana) and sexual activity (apart from foreplay) are all banned. Male nipples are OK, but naked breastfeeding is not. Photographs of bodily fluids (except semen) are allowed, but not if a human being is also shown. Photoshopped images are fine, but not if they show someone in a negative light.

Once something is reported by a user, the moderator sitting at his computer in Morocco or Mexico has three options: delete it; ignore it; or escalate it, which refers it back to a Facebook employee in California (who will, if necessary, report it to the authorities). Moderators are told always to escalate specific threats – “I’m going to stab Lisa H at the frat party” is given as the charming example – but not generic, unlikely ones, such as “I’m going to blow up the planet on New Year’s Eve.”

It is, of course, to Facebook’s credit that they are attempting to balance their mission “to make the world more open and connected” with a willingness to remove traces of the darker side of human nature. The company founded by Mark Zuckerberg in his Harvard bedroom is richer and more populated than many countries. These moderators are their police.

Neither is Facebook alone in outsourcing unpleasant work. Adam Levin, the US-based chief executive of Criterion Capital Partners and the owner of British social network Bebo, says that the process is “rampant” across Silicon Valley.

“We do it at Bebo,” he says. “Facebook has so much content flowing into its system every day that it needs hundreds of people moderating all the images and posts which are flagged. That type of workforce is best outsourced for speed, scale and cost.”

A spokesman for Twitter said that they have an internal moderation team, but refused to answer a question about outsourcing. Similarly, a Google spokesperson would not say how Google+, the search giant’s new social network, will be moderated. Neither Facebook nor oDesk were willing to comment on anything to do with outsourcing or moderation.

Levin, however, estimates that Facebook indirectly employs between 800 to 1,000 moderators via oDesk and others – nearly a third of its more handsomely remunerated full-time staff. Graham Cluley, of the internet security firm Sophos, calls Silicon Valley’s outsourcing culture its “poorly kept dirty secret”.

The biggest worry for the rest of us, however, is that the moderation process isn’t nearly secretive enough. According to Derkaoui, there are no security measures on a moderator’s computer to stop them uploading obscene material themselves. Despite coming into daily contact with such material, he was never subjected to a criminal record check. Where, then, is the oversight body for these underpaid global police? Quis custodiet ipsos custodes?

Facebook itself is guarding them, according to a previous statement to which the Telegraph was referred. “These contractors are subject to rigorous quality controls and we have implemented several layers of safeguards to protect the data of those using our service,” it read. “No user information beyond the content in question and the source of the report is shared. All decisions made by contractors are subject to extensive audits.”

And yet in the images due for moderation seen by the Telegraph, the name of anyone “tagged” in an offending post – as well as the user who uploaded it – could be clearly discerned. A Facebook spokesman said that these names are shared with the moderators to put the content in context – a context sufficient for Derkaoui to claim that he had as much information as “looking at a friend’s Facebook page”. He admits to having subsequently looked up more information online about the people he had been moderating. Cluley is worried that Facebook users could be blackmailed by disgruntled moderators – or even see pictures originally intended for a small circle of friends pasted all over the web.

Shamoon Siddiqui, chief executive of Develop.io, an American app-building firm that employs people in the developing world for a more generous $7 to $10 an hour, agrees that better security measures are needed. “It isn’t wrong for Facebook to have an Indian office,” he says. “But it is wrong for it to use an arbitrary marketplace with random people it doesn’t know in that country. This will have to change.”

In Britain, for example, all web moderators have to undergo an enhanced CRB check. eModeration, whose clients range from HSBC to The X-Factor, pays £10 an hour and never lets its staff spend too long on the gritty stuff. They wouldn’t go near the Facebook account. The job, says Tamara Littleton, its chief executive, is too big, the moderating too reactive, and they couldn’t compete on cost with the likes of oDesk.

So, if no one can undercut the likes of oDesk, could they not be undermined instead? If Mr Zuckerberg will not dig deeper into his $17.5 billion pockets to pay the street-sweepers of Facebook properly, maybe he could be persuaded by a little moral outrage?

Levin disagrees. “Perhaps a minute percentage of users will stop using Facebook when they hear about this,” he says. “But the more digital our society becomes, the less people value their privacy.”

Perhaps. But maybe disgruntled commuters, old schoolfriends and new mothers will think twice before sharing intimate information with their “friends” – only to find that two minutes later it’s being viewed by an under-vetted, unfulfilled person on a dollar an hour in an internet café in Marrakech.