Using Mechanical Turk for Fun and Testing
Amazon’s Mechanical Turk (http://www.mturk.com/) is billed as “Artificial Artificial Intelligence.” Amazon’s intent with the service is to connect tasks with workers who can perform the task. Frequently the task is one that is simple for a human but complex to program. One of the better examples is Amazon’s own iPhone application that allows you to take a picture of something, then creates a Mechanical Turk job to pay a person $0.10 to find a link to that thing (or the closest possible to that thing) on the Amazon store.
Other people have used Mechanical Turk for interesting little side projects, such as Andy Baio (of Waxy.org) who used Mechanical Turk to supplement data from Wikipedia in his quest to map out the years represented in the samples from this year’s Girl Talk release, Feed The Animals. Later, Andy wondered how much it would cost to get Mechanical Turk workers to take a picture of themselves and post it to the site. Someone else used Mechanical Turk to write a book about cats.
The particular problem I wanted to solve was testing the availability of a site from a wide range of locations. There are services that will let you do this, but they tend to be geared towards long term testing or simple screen captures from a limited range of IP addresses. I wanted to see proof that the site had loaded from 25 different locations around the world. My budget was $30.
First I set up the Mechanical Turk job. You can do these as “one-off” jobs or have code use the API to spit out repetitive jobs constantly. I went the one-off route and created a simple webform that asked 4 questions. The form looked like this:
Once the form was ready to go, I added $30 in funds to our account and told the Turk to offer the job out for people at $1 each, with each person only able to complete the task once, and with a 2 hour time limit. The time limit was because I was simultaneously capturing all the internet traffic sent to the website during the test, in case anyone reported an error. If so, I’d have a little more insight into exactly what problems were occurring. When the batch was finished, 16 people had taken me up on the offer. The results are delivered in spreadsheet format. An excerpt is below.
What we were able to show from the test is that people from 16 different IP addresses (spread all over the world) were able to hit the target website without problem.
