The computer showed a photo of what looked like a school board meeting.
My job was to rate it on a scale of 1 to 5 for 23 different qualities: “patriotic,” “elitist,” “reassuring” and so on.
I did the same for a photo of a woman wearing headphones — I gave her a 4 for “competent” and a 1 for “threatening” — and another of five smiling women flanking a smiling man in a blue windbreaker.
I submitted my answers. I checked the clock. Three minutes had passed.
I had just earned another 5 cents on a digital work marketplace run by Amazon called Mechanical Turk. At least I thought I had. Weeks later, I’m still not sure.
There are lots of ways to make a little money in this world. Amazon Mechanical Turk, which since 2005 has flourished, to varying degrees, in an obscure corner of the globe-striding behemoth’s empire, offers an uncertain, mystifying and often maddening way to make very, very little money.
On Mechanical Turk — named for a chess-playing “machine” from the 18th century that concealed a living chess master — scores of thousands of humans earn pennies or dollars doing tasks that computers cannot yet easily do.
It works like this: Employers, known as requesters, post batches of what are called Human Intelligence Tasks, or HITs, on Mechanical Turk’s website. A task could be transcribing an invoice, or taking part in a study, or labeling photographs to train an artificial intelligence program. (Occasionally a photo shows something disturbing, like a beheading.)
Freelance workers, known informally as turkers, race to grab and do the tasks, providing what Amazon’s founder, Jeff Bezos, once called “artificial artificial intelligence.”
Most tasks pay a dime or less, and there is a daily churn of tasks that pay only a penny.
And the weird thing is that workers — mostly American workers — will do them, for many different reasons.
People turk to save for a motorcycle. They turk to buy insulin. They turk to pay off debt or pass the time profitably while on the clock at a boring job.
Some do it because there are few decent-paying jobs that can be done at will. People who are confined to their homes by disability or social anxiety or who live where there are few jobs do it because, despite lousy wages, it seems like the best option.
Plenty turk full time. In a 2016 Pew Research Center survey of nearly 3,000 American turkers, a quarter said they made most or all of their earned income on the platform. More than half the turkers surveyed said they earned under $5 an hour.
Try Your Hand at Turking
Throughout this story we’ll give you the chance to do HITs (Human Intelligence Tasks) that The Times has devised based on ones that novice workers like me encounter on Mechanical Turk. Click Accept & Work on any of the tasks below to get started.
At the end of the story you can see how much you would have made and how long it took you. Please note: You will not actually be paid.
This task is based on actual tasks turkers complete on the Mechanical Turk platform.
As little as turking appears to pay on paper, in practice it often pays less because MTurk, as it is known, is a sloppy, shoddy free-for-all.
Turkers spend their time fighting requesters over an unfair 10-cent rejection or a missing 60-cent payment. They waste minutes filling out bubbles on defective questionnaires that cannot be submitted. They abandon “10-minute” surveys after half an hour.
They swap horror stories and warnings on turker message boards (“rejection on a $0.50 hit,” read one recent bulletin, “reason is ‘funds were not allocated’”). They leave scathing reviews on the turker-run site Turkopticon (“unfair and wild use of the rejection button”).
Just how much turkers make is the subject of considerable scholarly debate, but one paper published last year analyzed millions of tasks done by thousands of turkers. Though they probably overrepresented novice turkers like me who do the lowest-paying tasks, the paper’s authors concluded that if you count time spent looking for tasks and working on tasks that came to nothing, the median turker’s hourly wage was $1.77.
Only 4 percent of turkers, the researchers found, made more than the federal minimum wage, $7.25 an hour.
-->
SHOP NOW
Presiding over this production is the world’s biggest tech company, feet firmly planted on the sidelines. Amazon usually declines to get involved when turkers say requesters rip them off, even as it lets requesters hide behind aliases that can make them impossible to track down.
It has ignored turkers’ pleas to mandate higher wages, even as it takes a cut of each transaction ranging from 17 to 50 percent; a requester posting a 1-cent HIT pays one penny to the turker and another to Amazon.
Amazon even finds ways to recoup some of the pennies turkers earn, a reminder of the days when miners were paid in scrip redeemable only at the company store. While American turkers can get their wages direct-deposited, thousands of turkers overseas have only one way to get paid without incurring third-party fees: on an Amazon gift card.
While Amazon pays all of its American employees at least $15 an hour and favors raising the minimum wage, the company declined numerous requests to comment about the pay policy for turkers or anything else regarding Mechanical Turk. Minimum-wage laws generally do not apply to piecework jobs like turking.
Mechanical Turk is now one of a handful of big players in the field known as crowdwork or microwork. (One crowdwork company, Prolific, used by academic researchers, enforces a minimum wage: $6.50 an hour.)
Crowdwork’s proponents see a gleaming future — a borderless, no-overhead labor market where task-creator and task-doer meet at the intersection of supply and demand. Its critics see a throwback to something more Dickensian, where the lack of regulation and accountability keeps workers in the dark and on the defensive.
Mechanical Turk, in particular, combines the inconsistency and precariousness of gig work with Big Tech’s tendency to dodge liability for the icky things that happen on its platforms.
“This is a great little microcosm of what happens when you take away any rules and the wages fall to the bottom,” said Kristy Milland, a former turker turned labor activist who was one of the authors of the paper that analyzed turkers’ earnings.
This task is based on actual tasks turkers complete on the Mechanical Turk platform.
The bottom was where I dwelled during my brief tenure as a turker.
I tagged blurry, surveillance-looking photos of construction workers on job sites according to whether they were wearing hard hats or harnesses (1 cent per photo).
I helped train a virtual paralegal by describing a hypothetical injury claim (“I suffered a serious injury because a defect in my lawn mower caused it to start”) but earned nothing because the “submit” box never popped up.
I imagined myself as an “active investor” and rated the prospect of my supporting a Kickstarter for a device that makes nut milk “extremely likely.”
Over the course of several weeks in September, I completed 221 HITs in a little over eight hours of dedicated turking, and earned a grand total of $7.83. That works out to 97 cents an hour.
But hourly wages are not everything. Jane Lamont, a 30-year-old call center worker in Louisville, quit her $7.25-an-hour second job at McDonald’s to turk.
She works from 7 to 11 on weeknights, five or six hours a day on weekends, and typically makes $5 a day.
While the pay is “really low,” she said, she prefers turking to fast food for its freedom — freedom from having to wear a uniform, freedom to spend time at home with her mother, freedom to watch videos between tasks.
Besides, she said, she would probably be online anyway — “I like being paid while just being leisurely.” On weekends, she brings her laptop to her boyfriend’s house. While he’s wasting time gaming, she’s making money.
“I love it,” Ms. Lamont said. “I feel like that money I can spend on extra things that I want,” like two trips to New York City last year.
Katie Boehm of Pittsburgh turned to turking in 2017 after her husband, who has diabetes, lost his job and insurance coverage. Her own health issues keep her from working outside the house, and turking seemed like a lifeline.
She turks at least 50 hours a week, sets herself a minimum goal of $20 a day and usually makes $30 to $50.
Her husband’s insulin costs $1,500 a month. “MTurk covers about half of what he needs to survive,” Ms. Boehm, 40, said. “Silly insulin.”
This task is based on actual tasks turkers complete on the Mechanical Turk platform.
Amazon claims a turking work force of half a million, but independent researchers say the number of active turkers is smaller. An N.Y.U. data scientist who studies Mechanical Turk, Panos Ipeirotis, estimates that there are from 100,000 to 200,000 turkers, and that at any moment several thousand are doing tasks. The vast majority of turkers are believed to be in the United States — at least three-quarters, researchers say — with India a distant second.
Mechanical Turk was created to solve an in-house problem. In 2001, looking for help weeding out duplicate product listings, Amazon applied for a patent for “a hybrid machine/human computing arrangement which advantageously involves humans to assist a computer to solve particular tasks.”
The name was a homage to a contraption built by a Hungarian nobleman, featuring a bearded mannequin in a turban, that dazzled Europe with its chess-playing expertise. The moves were actually executed by a magnet-wielding human hidden beneath the board.
Mechanical Turk opened to the public in 2005 with considerable fanfare. “Market forces will define how effective it is for requesters and how lucrative it is for workers,” an Amazon executive, Peter Cohen, said at the time.
One requester I talked to has found MTurk quite effective. For a year, Ryan Schefke, the founder of a Texas company called Lead Liaison, has used turkers to transcribe business cards that salespeople collect at events. The contacts go into a database of sales prospects, a process known as “lead capture.”
Multiple turkers transcribe the cards to yield 99 percent accuracy, and the turnaround from the moment a customer scans a card is 10 to 12 minutes, Mr. Schefke said. A turker is paid 3 cents per card.
“Basically it’s set it and forget it,” he said. “It’s awesome.”
I mentioned that the one Lead Liaison task I did, transcribing a card covered in handwriting, took me 84 seconds, yielding an effective hourly wage of $1.29, and asked whether that was an issue for him.
“I don’t know why it would be relevant to us as a business,” he said. “The pay is per task, not hourly, and if a turker did one card and then went off to pick cotton in the field, that’s up to them. I actually think that’s pretty good wages. If you look at people in the Philippines or some other countries, it’s pretty good pay.” The minimum wage in Metro Manila is $1.17.
Many major corporations have availed themselves of Mechanical Turk. The New York Times Company has used turkers for at least three data projects.
MTurk also has big fans in the social sciences. Over 50,000 academic studies are conducted using MTurk each year, according to Leib Litman, a founder of CloudResearch, a company that helps researchers use the platform.
Turking is not always low-paying. The most proficient turkers make upward of $12 an hour, using computer scripts and free tools like HIT Catcher and HIT Forker to beat their fellow turkers to the sweetest HITs and racking up special qualifications that open the door to higher pay.
Dr. Litman said that a small cadre of super-turkers actually do most of the tasks on MTurk and estimated that the average wage for turkers who take part in studies run with CloudResearch at $6.50 an hour.
Amber Smoot is good at turking. She sits on the porch of her future in-laws’ house in Middleburg, Fla., and runs HIT Forker in the background while she watches her dogs play.
When the script catches a task that pays a dollar or more, her computer blasts a snippet of operatic rock and she grabs the HIT and sets it aside until she has a bunch. “I’ll queue ’em up and kick ’em all out,” she said.
Ms. Smoot, 32, doesn’t depend on MTurk to survive, but she knows that many people do. She spends a lot of time on message boards that offer tips to rookies.
“I love MTurk and I love the opportunities it gives people — it’s great for what it is, a side gig,” she said. “But the state of America is that people are turning to something that shouldn’t be a job and trying to make a living off it.”
Most turkers are relatively young — the majority who answered the Pew survey were under 32 years old — but some are older.
Jeff Archacki of Corvallis, Ore., is 60 and a former I.T. worker and computer store owner. Since 2014, he has gotten by on disability payments for a hand injury, and turking.
He sits at the computer from 2 a.m. to noon, doing crossword puzzles while waiting for good HITs, and typically makes around $40 a day. “I don’t live a lavish lifestyle,” he said, “but I’m not going to the food bank or anything.”
Ms. Milland, the turker advocate, said the biggest problem on MTurk was that requesters could decline to pay turkers by “rejecting” their submissions, but still keep the work. “I’ve known a requester to be open about the fact that they automatically reject 10 percent of jobs to pay for Amazon fees,” she said.
Turkers have little recourse in dealing with shady requesters.
Often, they have no idea for whom they’re working. Many requesters operate under generic, untraceable names like “Events” or “Panel” or “David” or “Josh.”
The task where I rated the patriotic quotients of photographs was posted by an outfit called Vision. I graded eight sets for Vision. At 5 cents per set, I should have made 40 cents. But Vision shortchanged me 15 cents.
I clicked “Contact Requester” and asked Vision to email me, twice. I never heard back.
What was I to do? According to a Mechanical Turk help page, it wasn’t Amazon’s problem. “If you have questions about the instructions in a particular HIT, completing a HIT, or why your HIT was rejected, contact the requester,” it says. “Please note that requesters determine when your HITs are approved.”
For another requester, I transcribed handwritten disability findings. One contained the patient’s name, describing him as “fully disabled” and “unable to sit/stand for more than 90 minutes w/o pain.”
Why was this presumably confidential patient information posted on Mechanical Turk, a platform open to anyone with access to an email address?
The requester, p9r, did not respond to my “Contact Requester” query.
Searching the p9r website for direct contact information, I found only the physical address of a public high school in Portland, Ore. A receptionist and a business teacher there said they knew nothing of p9r. Weeks later, p9r wrote back but, when I said I was a reporter, I was ghosted again.
(Amazon has made one recent effort toward transparency. Since July, it has listed requesters’ approval rates — valuable information for a turker who is deciding whether to do a task.)
This task is based on actual tasks turkers complete on the Mechanical Turk platform.