Turk Work

A cloud showing word frequencies does not begin to do justice to their expressions, but all tools for seeing have that which they highlight and suppress so I wanted to cut the posts in different ways.

Two Turkers suggested creating a union. Seven mentioned some sort of minimum wage. But ambivalence surrounds the Turk work’s flexibility which some people linked to the lack of protections. One Turker says that if they get injured on the job, they probably had it coming. Another says that Turkers aren’t employees the same way the self-employed are.

“Each worker would have the right to work on many different types of hits.
Each worker will get paid for amount plus any bonus offered.”

“To amke sure that workers are not unfairly rejected, put some type of appeal system in place so the worker has someone to turn to.”

“If I could make the Mechanical Turk Bill of Rights, then I would make sure that workers were compensated for any work that they tried their best on, and such work could simply not be rejected. I realize that if someone obviously didn’t take the task seriously, then, perhaps their work should be rejected, but if they honestly tried their best, then they should be rewarded for their time and effort. Because some requesters using Mechanical Turk simply aren’t good individuals, as they will review one’s work, reject it, provide an extremely lame excuse as to why, even though the person who submitted the HIT was right on par, just so that they (the requester, that is) can get their work done, without having to pay a penny. In my opinion, this is extremely immoral and should be frowned upon by those at Amazon. Secondly, I think that the reward that workers are offered should correspond with the work that the requesters are asking them to do. For instance, while some tasks offer a penny because they really are truly simple and not that time consuming, those are okay in my book. But, when a requester asks for work that is going to take a significant amount of time and requires a great deal of effort, a penny will simply not suffice! And, I would also change the amount of days that a HIT has to auto approve to 7 days instead of 30. Right now, if someone does a HIT and submits it, the requester has 30 days to review it and decide whether or not to approve or reject it. And if the requester doesn’t look at the HIT during that 30 day period, it is automatically approved. However, 30 days is really too long of a wait, especially when someone is waiting for tasks to be approved that will only pay you less then a couple of dollars! Lastly, I would also like workers to have more of a say around here, so that they can not easily be taken advantage of, and are treated fairly, as they should be. Amazon seems to pay more credence to the requesters, simply ignoring the fact that without workers, nothing would be done! However, since some workers would be willing to do difficult tasks for pennies, other workers that know this is not a fair trade can’t really make a statement. And this is why some type of organization or union needs to be formed to ensure that we are treated like they should be, and not like people working in third world country sweat shops!”

“MT workers should be assured that they are not penalized for returning hits that do not work correctly.  Often workers are not aware that a hit is defective until they have already accepted it.  Workers are forced to either return the hit or leave it incomplete - both of which affect their ratings.  The only way to get this reversed is to contact the requester and, in my experience, most requesters are not very responsive to these types of inquiries.”

“1. There must be an option to correct rejected hits.
2. Time limits may not be assigned to a worker.
3. A requester must answer any messages a worker sends.”

“It should include intellectual property right and specific duration in which cheques or money will transferred.”

“I would add a way to dispute rejections and try to make approval faster.”

“Mturk workers should have some type of legal recourse against requesters who use the site only to get free work. No one ever sees WHY they were rejected, so it’s totally up to the discretion of the poster. However, it’s discouraging to work hard on an article for a mere $4.00 and then to be rejected for some unknown reason. There are posters who do this over and over, even if a worker seldom gets rejected by the majority of posters. I had this happen to me, and when I contacted mturk I was told that there was nothing they could do and that I’d have to just mark the requesters posts as inappropriate if I saw them. I’ve been fighting the guy ever since, and it’s been several months now. We should get to see a requester’s rejection rate prior to doing work for that person. We can be rejected, yet the requester still has our articles and sentences that he can use anyway. That is not fair!”

“That Amazon Turk Workers will receive payment for each completed and approved task within ten business days following completion the HIT.”

“There should be something to allow workers to confront jobs to find out why a job is rejected.  There is email, but it is usually not answered when questioned.  I sometimes wonder if larger paying jobs ever really plan to pay in the first place.”

“1. All Requesters must approve/deny in 48 hours or its automatically approved
2. All HITs that are more than a paragraph in length (or 40 words) cannot be worth less than 5 cents
3. All rejected HITs must have reason for denial on them
4. All bonuses must be properly labeled”

“All submitted HITs must clear in three days.
All requesters who do not pay for HITs within three days are banned from future requests.
No HIT shall be spam or spam-like.”

“1. Better and fair payments.
2. Other payment methods: Paypal, Moneybookers, etc
3. Access to non-american workers and providers
4. Laboral rights respect
5. Ownership of intellectual rights to the turkers”

“We the Turks, in a world that requires productivity in working together, will work honestly and diligently to perform the best work we can. The Requestors, in turn, will provide useful work and will pay us fairly and quickly, providing bonuses for especially good work. The goal is to create a working environment that benefits us all and will allow us the dignity and motivation to continue working together.”

“Those who create hits should either approve or deny based on the work and should NOT be allowed to post a new hit asking you to approve someone else’s previous work.  It is unfair to the person who completed the original hit.”

“Allow workers to see their rejected work.”

“The right to receive feedback when a qualification is changed.”

“Workers should be able to protest rejections on jobs that are worth $1 or more, as long as they have a valid claim.”

“There should be a time frame for which requesters have to pay the worker.”

“I think that MT workers should have the right to receive answers if they contact requesters.  I also believe that we should have a chance to correct small errors in large projects before a requester arbitrarily rejects the work.  I once missed a few typos in a large project that took hours to complete; rather than asking me to correct it and even though I requested a preliminary review and offered to make any needed adjustments when I submitted the project, the requester rejected the work completely.  Because of that experience, I no longer complete time-consuming projects on MT.”

“Legally speaking, the failure to offer MT’s what they offer employees is a good thing. We operate as independent contractors, not employees. Were we employees, they would set the hours one worked, assign HITs as they saw fit, take out taxes, sit on the funds until payroll time, etc. ad nauseum.

What I would like to see is a requirement for disclosure of privacy of information when completing surveys. E.g. If I answered all of Ask500People’s offering and they kept track of my responses (identified with me personally vs. anonymously) chances are I would never complete their HITs. We shouldn’t have to e-mail to find out - it should be a required disclosure prior to accepting the HIT.

I would also like to see some sort of minimum of what can be offered based upon a realistic expectation of how long the HIT should take. The HITs offering a penny for 15 minutes worth of work enrage me (esp. when there are thousands of that HIT available) I don’t have a problem with different categories for payment (e.g. for fun/non-profit/student research shouldn’t have to fall under the same minimum). Keeping in mind work is completed in economies/countries other than the USA, I’d even be willing to accept a low expected wage (say $3 a hour as the base) but these folks offering less then $1 an hour are thieves as far as I’m concerned.

Too, I would like to see the ability to return a HIT as defective so it dings the requester’s reputation and not mine. Let’s face it, if I’m supposed to find an item for sale on Amazon but they show me a child’s crayon drawing … there really needs to be a way to handle that without it altering my numbers.”

“I would like to be able to know why a hit has been rejected….as in where did I deviate from the directions. Some requestors will just say nope you did it wrong. How am I supposed to get better if they dont tell me how to improve”

“To more readily be able to contact a requester when they reject a hit you submitted without giving an accurate answer as to why it was rejected. To not have hits that were not properly compiled by the requester count against you… I mean come on, some requesters set the time allotted to zero seconds or write the instructions incorrectly so that the work they receive did follow the given instructions but was not what they were looking for. Those should not change our rejected rate. Also, requesters who do not pay out for their hits within a week should be subject to ratings the way we are.”

“I would make sure that workers are not penalized if audio or video does not play correctly. I would create a minimum wage of no less than .03 per hit”

“1. Right to dispute rejection”

“1.  MT worker has the right to defend rejected answer(s)

2.  Hits will be labeled clearly and completely to avoid confusion as to subject

3.  Once a hit is accepted you will not be given “”this hit cannnot be accepted”"

4.  Hits will be catagorized to make scrolling thru hits easier”

“MT workers shall receive a minimum of 0.01 compensation on ALL HITS completed; with accepted HIT resulting in worker receiving full pay as promised for acceptable work on each HIT.
Workers shall receive payment for completed HITS within 7 business days.
If a worker completes many hits for the same provider and a few of the HITS are rejected; worker shall receive feedback on at least one identified, rejected HIT explaining why it was not acceptable.”

“The right to dispute rejected work.
The right to review of work within 48 hours.
All work should pay minimum of 10 cents.
Should not have to download anything.
Should not have to log onto anything. “

“Instructions will be clear and specific. Workers will be paid for work that is performed correctly within the instructions given. Workers will be paid promptly for their work. If work is rejected, workers will receive an explanation. Workers will have the right to appeal a rejection of their work. Workers will also have a way to ask for clarification of instructions. Workers will be able to view the approval rate of requesters.”

“Although it is rare, I have encountered unethical Requesters who have rejected your work without proper or truthful explanation, which one has to assume is simply a way to get some work done for nothing. Especially irritating when you
know the work was done correctly. In the couple of instances where this has occurred, the Requester never had the courtesy to reply and substantiate, needless to say. Therefore, hits should be reviewable by an Amazon arbritrator
if a rejection is challenged by a worker. Where evidence shows obvious abuse to
defraud workers, that Requester should be banned from posting hits on this site.”

“”Exactly what is in the provider policy. If I don’t like a requester, I don’t do work for the requester. If I have a problem giving away the rights to my work for all eternity, then I don’t give my work away. And if I manage to injure myself doing a HIT, I probably had it coming.

We’re independent contractors. It happens.”

“I feel that we should be able to comment on the people that have posted requests.  I have had a couple hits that I have completed but then the person did rejected the hit.  I feel that this should be tracked since some are using workers and not paying them”

“If I could make a MT worker bill of rights, it would state that all worker will be paid for their work fairly. It would also state that workers should not be paid any less than $0.25 for each turk.”

“I think that a worker’s work should be approved ASAP. There are some requesters like Enrovia Research who take a long time almost over a month to approve a work. This leads to a confusion whether to work on their further HITs or not. I believe that the time for approving /rejecting an HIT should be reduced considerably.”

“The right to timely acceptance or rejection of hits. After 10 days aa hit must be reviewed by a requester. If it is not reviewed, it is accepted and the worker paid.”

“All requesters would verify finished HITs and provide payment in less than a week.”

“that requestors must accept or reject HITs within a certain amount of time”

“I think that if you have to return a hit due to circumstances beyond your control such as, pictures not downloading completely, that you should not be penalized for the return.”

“-Guarantee that we won’t be asked to do work that is spam or that requires us to spend money
-Guarantee that we will be paid a fair sum for our work
-Guarantee that we will be paid soon after finishing the work, and that work will not be rejected withou” [[Editors note: The rest of this didn't come through.]]

“‘all ownership rights, including worldwide intellectual property rights, will vest with the Requester’
Well I agree with this one

‘you will not be entitled to any of the benefits, compensation in the event of injury’
I also agree and mostly don’t care about this one

There’s no point in having a bill of rights if there’s no one to defend them.

This might sound a bit crazy, but here’s just an idea. I believe that there should be an organization/website that will defend a particular worker’s interests, some sort of a MT workers’ union. It could also be extended beyond MTurk to accept members from other crowdsourcing communities and help them solve their problems.

It should gather votes and fees from the union workers and try to negotiate their demands and needs directly with MTurk staff sometimes by even announcing a strike of all union members at MTurk for a period of negotiations. And there should be compensations for union workers for not working on these days, and penalties if they do try to work.

The union could also help with introducing minimum wages on MTurk, maximum working hours limit, restoring an account banned by mistake, even doing an independent investigation of logs of all the interaction between requester/worker over a particular HIT to resolve conflicts.”

“All work will be approved or rejected within 12 hours.”

“First, a private message to the poster- terrific hit idea and I’m doing this for the principle, not for the $0.10. Because of how important I deem some of the things that I have to say, I’m combining a Declaration and my Bill of Rights. I apologize that my formatting is not more akin to the BoR format, but I’m so frustrated with MT that I’m considering no longer using it at all.

1) The owners of the site (Amazon) need to set up a review board for rejected  hits, for a variety of reasons.
1A) Many HITs require two out of three workers to agree and only the two that agree get paid (unless all three agree.) If three people are doing a HIT that compares two things and two say that “”Apple”" is identical to “”Giraffe”" and only the third gets it right, who should be paid? Under the current system the only one who got it right has no recourse. (I no longer do these.)
1B) Some HITs may require users to write a sentence or ask a question using keywords. The HIT will be rejected if it contains potentially offensive language or ideas. Unfortunately, the articles or keywords are often quite offensive and there is no adult content warning. How can I ask a question containing keywords that must be used, yet disqualify the HIT?
1C) Some HITs with larger payouts, i.e. several dollars get rejected for no reason and the requester will not respond to e-mails. I once had a hit rejected because the hit required me to sign up on another site (already a violation) and complete something, then wait 24 hours. I did this before accepting the HIT, because the HIT had a one hour time limit. I got an e-mail from my referrer (requester) thanking me for joining. Two days later I accepted the HIT and was rejected because he claimed that I didn’t wait 24 hours between accepting and submitting the HIT! It had a one hour time limit. I forwarded him the e-mail that he sent me and got no reply.
2) The worker should have access to HITs that were submitted and rejected to see why they were rejected and whether they did something wrong or whether one of the situations outlined above is true and they should submit it to a review board. (The site charges fees for requesters, so they should pay a review board.)
3) Workers should be paid promptly. A month should be the outside limit.
4) Requesters who submit inappropriate HITs, such as asking workers to sign up for one of those scam sites where you have to complete bronze, silver, gold, and platinum offers to “”win”" an X-box or hi-def TV should be banned after sufficient review. (The requester who scammed me in 1C above keeps submitting the same HIT under different names, so I spend my first 10-15 minutes after I log on to the site finding his HITs and reporting them as inappropriate. They do violate the rules about requiring sign-ups for other sites and nobody will ever get paid by this guy.)
5) HITs should have a reasonable time limit. Earlier this week I spent too much time carefully wording and reviewing a HIT with a survey with eight essay questions and a three minute time limit. It abandoned me when I was on question #8. I couldn’t redo it, because I had already accepted it.
6) Requesters who frequently reject HITs for potentially offensive content should be required to have posted guidelines via a link on the HIT. If you want me to “”ask a question”" about a “”transvestite hooker dealing drugs”" I’m wise enough to return it because I know that it can’t ever be on topic and meet your guidelines. I’d still like to know those guidelines and why my HITs are rejected for using your own keywords. New users might also benefit because they may ruin their approval rating without knowing about your double standard.
7) If a worker is going to be banned for doing the same HIT twice, it should be set up like the HITs that won’t allow you to do them twice, i.e. the survey that abandoned me that I couldn’t retake because I had already accepted it. Requesters have every right to limit the number of times a user can do a HIT, but have an easy tool to do so without hurting people who do hundreds of HITs and can’t remember which ones they did last week.
Workers should be allowed to review requesters a la E-bay. Each requester name should be linked to a page displaying their acceptance/rejection rate and worker comments like “”Fast pay,”" “”Thanks for the bonus,”" or “”Rejected for no reason.”"
9) Requesters should not be allowed to require a higher acceptance rate than their own acceptance rate. If you reject half of your HITs, why should I need a 95% approval rate to work on them?
10) You should be able to save your work in progress on longer HITs. I have spent an hour on a HIT only to have to start over because the site that displays the HIT went down. It wasn’t my connection because I could still access all the other internet sites that I tried.

I can think of several more things, but the original Bill of Rights had ten, so I’ll stop here. Thank you, Lilly C. Irani for creating this HIT that allows me to express my dissatisfaction with the site in its current form. I have a “”real”" job and do this for fun/extra books. It’s disheartening to have your work rejected for something as simple as claiming an “”Apple”" and a “”Giraffe”" are not identical. I don’t care about the penny I didn’t earn for knowing the difference between an apple and a giraffe, but I’m angry that MT will take requester’s money but not manage, oversee, or mediate the problems and injustices on their site.
Thanks for the time and your excellent HIT.”

“I wouldn’t change anything. I like the way it works, because if there’s a hit I don’t want to do…I don’t do it. If I want to, I do. It’s a simple system and I like the way it is.”

“Guarantee of payment- how is it that amazon has been scammed by requesters who approved work but just didn’t pay up?

Third party review of approvals”

“Turkers are not employees in the same way self employed people are not employees.
You choose what you want to do no one is forcing you to do this.”

“1. When HITs are denied without cause, AMT will take action so that workers are not exploited.
2. That AMT enforce policy. HITs requiring membership or disclosure of email addresses be removed from the site by AMT, rather than ignored.
3. That AMT more closely monitor contests, such as NowNow to make sure cheating and/or manipulating isn’t occurring.
4. Don’t know if this is true or not, but someone indicated that AMT takes 90% of what a requester pays, and only gives workers 10%. This would by all means explain why HITs pay so low, most days you can’t even make $1. an hour. If the 90/10 split is correct, there needs to be a major adjustment where workers are by far better compensated. I don’t know anyone at AMT in Seattle that would be willing to work for $1. an hour. It is nothing short of exploitation. “

“More opportunity for details as to why work was rejected.
Faster processing of work that may sit pending for weeks. “

“Requesters would be required to accept or reject a task within a reasonable amount of time–say, a week–or Turk would automatically pay for the task out of a deposit account set up as insurance for Turk workers.  The reason I say this is because there is a task I completed on September 19 that still hasn’t been paid or rejected.  I emailed the requester asking why it’s been ignored and have yet to receive a response.  C’mon, gang, a month’s wait is ludicrous for this kind of money.”

“1. One must be fairly compensated.
2. If the worker completes the task and the buyer does not pay within a reasonable period of time, it will start to accrue interest.
3. Due Process: Hit rejection is not allowed, without fair trial from it’s peers.
4. An MTurk worker must know his/her rights. The Bill of Rights, must be displayed on the website.”

“The right to review rejections.
The right to chart bonuses.
The right to see favored HITS separate from the big list.”

“Above everything else, we should always be entitled to knowing the reason why some HIT we’ve submitted is rejected. Otherwise, people can simply reject them, ‘just cause’.”

“I am concerned about the level of pay. All MT workers should be guaranteed a minimum pay for a set of tasks. It would also be good to make the long term requesters like ‘CastingWords’ bring in some policy of recognizing dedicated workers and offering them yearly bonuses.”

I’m concerned about  the fact that as a Turk worker I can complete a HIT that requires considerable time and effort and must submit the results to the requester and have no control of my work product.  Does the requester receive the actual product whether or not they approve and pay for the work?

“I would provide that there be a right of appeal when a provider denies a large number of HITS.  I would also make it required that bonuses be listed and, in a clear, obvious way, that a worker be able to tell who paid it.  I would make it possible to determine a providers rejection percentage.”

“I think that there should be a firm time limit on when we get paid for jobs.”

“If a hit  is rejected, turker has the right to know why and have it explained to him.
All hits that are not rejected should be payed.”

“workers should get paid within 2 days for submitted hits that are acceptable
there should be a $0.05 minimum on hits”

[[Editors note: I paid 10 cents for this HIT (human intelligence task)]]

A few things that I would have on an MT worker bill of rights would be a) hits should be approved/rejected within 48 hours.  There is no reason why a worker should have to wait for 29 days to find out if they will get paid for their work; b)rejected hits should have a viable reason for the rejection; c)worker’s emails should not go unanswered or be ignored by both those posting hits and by mturk staff;

“1. An MT worker has the right to challenge a declined HIT - especially on writing assignments.
2. MT workers have the right to be protected from unscrupulous HIT requestors who do not accept properly completed HITs.
3. MT workers have the right to post complaints on a public forum.
4. MT workers have the right to demand redress for copyright infringement against requestors who decline a HIT and then steal the worker’s work.”

“MT worker has the right to learn and express their point of views without ever being censored or belittled.
MT worker has the right to receive the payment for their works no later than 2 days after the works have been submitted.
If the work is rejected, MT workers have the right to know the reason for the rejection.”

“Turkers would have the right to know why their hit was rejected and have the ability to make a case against the rejection if they think it was unfairly rejected.

Turkers should have the right to pause the timer on hits for a certain amount of time before it started up again.

If a turker returns a hit, they should be allowed to re-accept it again, but if they return it again, then the hit will no longer be available to them.

There should be a requester directory for common requesters who offer hits a lot. It would explain who they are, give some background on them, etc.”