Crowd Design: Engine of Innovation

The area of crowdwork has developed so swiftly that we’re spoilt for choice in terms of crowdsourcing platforms. Irani and Silberman [5] offer an analysis of one. Born in 2005 – Amazon Mechanical Turk (AMT) acts as a meeting point for requesters (employers) who want to outsource jobs split into micro tasks, and workers, also known as Turkers (politically incorrect as this may seem), who are usually paid to complete these tasks. AMT enables companies and researchers to crowdsource their work. So, on the surface at least, it is an amazing match-making place where jobs can be done in a matter of hours, giving workers flexible working-hours and choice of task to complete.

AMT takes its name from the Mechanical Turk, a chess-playing automaton which faked the ability to compete with humans [3]. But unfortunately, AMT borrowed the principle as well as the name. In the 18thcentury, Kempelen – creator of The Turk, had a chess master hidden inside the automaton performing the job that machine could not [ibid]. In the 21stcentury AMT have managed to make the workers – actually the whole crowd that performs the job, invisible. It is this invisibility that Irani and Silberman [5] criticised in their paper.

So, how do you ‘hide’ the whole crowd? Amazon is doing a much better job at representing requesters on their platform than workers: workers’ ratings are used to filter and monitor access to jobs, requesters can decide whether they will pay for completed tasks with no obligation to say why, and with Turkers being contractors there are no minimum wage requirements [5].

Many have raised ethical concerns in terms of this requester/worker inequality [2, 5, 6]. Irani and Silberman [5] were among a few who decided to respond to this concern by creating Turkopticon. Turkopticon is a browser extension for Firefox and Chrome which is aimed at enabling workers to assess requesters and provide feedback of their experiences; thus, making requesters responsible for their actions and providing support to each other.

Turkopticon can be seen as an example of Scandinavian participatory design advocating for the empowerment of workers. The tool goes beyond a simple website with reviews and acts as mediator allowing people to be brought together around common matters of concern despite differences in Turkers’ perception of the AMT ecology. Turkopticon builds publics.

Ironically though, its creators seem to see their creation as something which has inadvertently, mainly thanks to media representation, depicted workers as helpless drones and themselves as ‘saviours’[4]. Easy to understand how this might happen, where focus on concepts like ‘digital sweatshops’ have fostered the idea that creativity is alien to ‘low-skilled’ Turkers.

In a way, the Turkopticon creators have echoed Björgvinsson et al.’s [1] idea of democratic innovation by creating controversies which do not only build publics but sustain and evolve them. They have created a Thing, initially an artefact that after being made public and given to its participants, designers in their own right, thereby raises concerns and possibilities for further exploration (e.g. expanding the functionality of Turkopticon [4], building the Dynamo activism platform with Turkers [6] and involving them in conferences [4]).

So is Turkopticon a kind of Frankenstein’s monster, something designed to render workers visible which contributed further to their invisibility? Maybe, but only, I would argue, if we see this monster as a necessary one, creating controversies and friction, providing workers with identity and ultimately stimulating change or at least debate. To stretch the analogy a little farther, their creature is an evolving one, an engine of innovation.


[1] Erling Björgvinsson, Pelle Ehn, and Per-Anders Hillgren. 2010. Participatory design and democratizing innovation. In Proceedings of the 11th Biennial Participatory Design Conference, 41–50.

[2] Ilka H. Gleibs. 2017. Are all “research fields” equal? Rethinking practice for the use of data from crowdsourcing market places. Behavior Research Methods.49(4), 1333–1342.

[3] Jeff Howe. 2006. The Rise of Crowdsourcing, Wired, 14(6), URL (accessed 6 November 2018):, 1-5.

[4] Lilly C. Irani, and M. Six Silberman. 2016. Stories We Tell About Labor: Turkopticon and the Trouble with “Design”. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI 2016), 4573-4586.

[5] Lilly C. Irani, and M. Six Silberman. 2013. Turkopticon: Interrupting Worker Invisibility on Amazon Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013),611-620.

[6] Niloufar Salehi, Lilly C. Irani, and Michael S. Bernstein. 2015. We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI 2015), 1621–1630.

Author: Irina Pavlovskaya

One response to “Crowd Design: Engine of Innovation”

  1. Matt Wood says:

    Excellent stuff Irina, you situate your discussion of crowdwork very nicely in the AMT, and provide a great balance of description and critical reflection. I particularly like how you drew on the parallel of the invisible chess master and the invisible crowd, and your critique led on very well to the Turkopticon. You suggest this is a good example of scandinavian participatory design (‘participatory’ at it’s highest order, arguably!) – but the authors don’t explicitly label it as such, so a little more reflection around what made this work participatory might have been well placed. The nature of how Turkopticon was designed might be interesting to discuss in relation to the problems you identified. Having said that, the insights you draw out are very well placed and well evidenced, leading to a very satisfying conclusion. Well done again Irina!

Leave a Reply

Your email address will not be published. Required fields are marked *