New Blog

I decided, for the sake of organization, to start a new blog which will focus on grad school related topics not specific to this research project. I will keep updating here with progress in the ESL HIT (coming shortly).

Follow me if you want to get spammed with information about my work and life at Penn. Don't follow me if you just found this blog from Googling an MTurk error that is probably still unsolved.

http://couldnotresolve.blogspot.com/

Tech Report

Finally...after having attempted to do it at least once a week since June...I have written up a technical report of my work this summer. If you read my long post last month, and just found yourself starving for more details, then you are in luck. There is (always) more information that I would like to add, but this is a good walk through of the work we've done on the ESL HIT this summer, and the state of affairs as of now. Of course, as I was writing it, I realized how much of it (especially re: Quality Controls and Annotation) is about to be redone and will need to be rewritten. But I suppose having a constant stream of things to change is much better than having stalled out.

So enjoy: ESL HIT Technical Report

MTurk ApproveRejectedAssignment HowTo

As I have mentioned previously - I had a traumatic first experience trying to QC my workers. Out of the 200 assignments I had posted, I rejected about half of them-- nearly all of which were good quality work. The responses from Turkers ranged from anger ("35 rejections?! Really!!?") to sad confusion ("I just want to know what I did wrong...") all of which made me feel terrible.

Luckily, MTurk recently added a API Call to undo accidental rejections. Chris and I reworked some sample code from the Java SDK to make the call, which you can download here. It is very straightforward to run (there are instructions in the tar).