From the Precision of Electro-Mechanical to the Ambiguity of Socio-Technical
What began with research on the interactions of individuals with computer technology, transformed to the social, and now also society. From the precision of one, to the ambiguity of the many, from a universal machine to a universe of possibilities.
Early computing pioneers were very much aware of computational limits, recognising the pivotal role humans have in the overall system. With an innate human preference for shiny new things, initially research in Computer Science and later Human Computer Interaction (HCI) understandably concentrated efforts on the machine side of these systems, rather than the human and social aspects.
The move to the social, especially Suchman’s work on Plans and Situated Actions saw a shift in emphasis, with greater influence given to the human side and context of the interaction. The seeds of co-production appear in Suchman’s work, contributing to our current research, as well as the turn to the civic where it is recognised citizens can collectively contribute to social justice, including how local services are provided.
The interdisciplinarity of HCI is a natural fit with the multi-complexity of society – neither is a homogenous mix. HCI research has benefitted from looking to and connecting with the alternative, the niche, the outliers. Socio-technical research cannot be defined with precision; we cannot assume what problems need to be solved, nor what the solutions will be.
Future research should keep open as many avenues as possible, consider what we may have overlooked in the past, and seek outliers and oddities. Our research should not attempt to follow well-worn paths nor focus on the shiny new. It is suggested that examining the mundane, the ordinary and the simple may provide worthwhile insights – big things have small beginnings.
Photographs: Author’s own taken at The National Museum of Computing, Bletchley Park. The electro-mechanical (left to right): reconstructed Turing-Welchman Bombe machine wiring; Tony Sale with his rebuilt Mk 2 Colossus; reconstructed Turing-Welchman Bombe machine drums.
Multiplicity of Perspectives
The foundations of HCI are not that much younger than the foundations of Computer Science itself. The interdisciplinarity of HCI mirrors the story of people central to computing’s journey, many of whom were unconventional, alternative, outsiders and somewhat controversial within the societal norms they lived in. This acceptance of the unusual and different perspectives has helped drive innovation in HCI, and in turn this can be used to foster innovation in society.
My own first encounters with computers were learning to program at school aged 13. This incorporated using a sharp pencil to write BASIC on coding sheets, which were sent to the University of Edinburgh for keying onto punch cards and submission to “the computer”. The printed output a week later was disappointingly often “Syntax error at line 10”.
From the earlier times of remote interactions, HCI has been transformed. The HCI field has broken out of “disciplinary boundaries” (Shneiderman, 2011) to become an interdisciplinary area with a dynamic and fluid mix of researchers, with backgrounds in computer science, information studies, psychology, sociology, political science and art. It is very like design which Manzini (2015a) says is a “third way of doing things between hard science and human science”. Carroll (2008-2018) writes HCI “is the intersection of the cultural, the social, the cognitive, and the aesthetic with computing and information technology”.
So how did HCI get to where we are now?
In the Beginning
We have come a long way. Charles Babbage’s interaction with computational machines was very much physical, with the construction of the analogue machines using mechanical parts. His colleague Ada Lovelace published theories (Lovelace, 1843) which are now recognised as the first computer algorithms. Lovelace’s insights may not have been possible without a broad interdisciplinary background. She was a mathematician, an engineer, a writer, an illustrator, and… a computer scientist. Lovelace’s contributions were largely overlooked until more recently, but opinion is that she had greater insight than Babbage and “went beyond number-crunching to see possibilities for wider applications” (Füegi and Francis, 2015).
Babbage might be known as the “father of computing”, but it is Alan Turing who is recognised as the “father of the modern computer” (Hodges, 1983). Wartime efforts spurred on developments in computing. Turing had little concern for status or class and privilege structures (Hodges, 1983). With a wide-ranging background in in science, maths and philosophy, his research work’s considerable legacy included the concept of a universal machine, and also the absolute limits of the computable.
In those initial years of computing, the interface between humans and computers soon started to change from electronic rewiring to printed or punched paper, but interaction was still limited to programming, operating the hardware, and utilising the output (Grudin, 2012).
Christopher Strachey, another outsider at the periphery of traditional society, was invited to the University of Manchester by his friend Turing. True multimedia output may first have occurred in 1951 when Strachey programmed a draughts game which played the national anthem on termination. Recently restored recordings are the earliest known computer-generated music. Strachey was the University of Oxford’s first Professor of Computer Science. This is where I later studied a computer science master’s degree conversion course, in the early days of the public internet.
In 2016 Hodges invited me, as a friend, to his Strachey Lecture – The Once and Future Turing. I was intrigued to see the inclusion of a photo from the Turing Archive of a computer-printed personal message, annotated and sent by Turing to his associate and close friend Robin Gandy, demonstrating early “consumer computing”. During the same lecture, Hodges explained in his discussion of the decision problem (Turing, 1936), that Turing “invented the mentality of computer programming” from the rigour of mathematical foundations, and that he was not thinking just about the mind of the programmer, but analysing “the action of the human mind and what it is a person is doing when they are computing”.
Early formative steps in HCI came from the fields of research into human engineering, human factors and ergonomics, where the emphasis was on efficiency of action. What is seen as one of the first papers in the HCI field, but well before the term “HCI” was introduced, Shackel (1959) describes how ergonomic design principles can be used to reduce mistakes when “[re-programming] from one problem to solve another”.
With the development of vacuum-tube displays, screens provided greater interaction flexibility, and the humans interacting with the computers expanded beyond individual mathematicians and technicians. As we shall see later, this expansion of the “user base” was, and continues to be one of the main driving factors in the changing scope of HCI research.
From One to Many
Even in the early years, visionaries like Nelson (1960) foresaw the vast inter-connectedness we have today and Engelbart (1962) contemplated human augmentation by machines. In 1965 it was Nelson who introduced the term hypertext, and twenty years later I became a member of the Edinburgh Home Computer Club and first went online, joining Micronet 800 and other services on BT’s Prestel videotext service. It was almost another ten years when in 1994 I created my first web home page, something which afterwards turned into a commercial career.
What began primarily as research into user interaction design, HCI has changed, morphed, and reinvented itself many times. Invigorated by its interdisciplinarity, the field comprises an ever-shifting cornucopia of viewpoints and visions. From a human factors/engineering approach, this shifted to a cognitive approach drawn from psychology, then to alternative cognitive methods, more social approaches including situated action, ethnomethodology and ethnography, and then drawing theory from a wider range of fields, including activity theory and grounded theory (Rogers, 2012, and Harrison et al, 2007).
These viewpoints and visions have also reflected the changing role of the “user”. Initially firmly rooted in the consideration of human factors and ergonomics from an operator’s point of view, this changed as the outputs of the machines were driven by economic needs to management information systems, where the understanding of typically printed output became a more prevalent driver. Cognitive and design approaches developed, drawing on experience from other fields, to inform how software is developed (Carroll, 2003) – the user as a programmer.
Driven by the vast expansion of computing in the 1980s into workplaces, the home and the mass consumerisation in the 1990s and 2000s, there was an increase in ideas drawn from social sciences, with less emphasis on traditional cognitive approaches. This change to the scale of “user” affected research perspectives. HCI had originally focused on individual interactions, but this has expanded to include groups of people, and more recently to vast numbers of people (through ideas like ubiquitous computing, social computing and the growth of the data society). In a constructivist approach both the past (knowledge) and current situation impinge on our response/interactions and societal-scale groupings can multiply this effect.
Consider one theory from the late 1980s which examined how the context of the interaction between humans and technologies leads to intelligent action. Suchman (2006) wrote, in a discussion of critiques, that the term user “singularizes what is actually a multiplicity and fails to differentiate actors with very different relations to a given artifact”.
The Social with Plans and Situated Actions
Suchman (1985, 2006) touched on a topic close to Turing’s previously mentioned consideration of “what a person is doing when they are computing” (1936) which resonates in Turing’s later work on intelligent machines (1948).
Suchman studied photocopier users as they tried to assist each other. In Plans and Situated Actions (P&SA), she challenged the genius role of plans in human-computer systems highlighting that the “attribution of knowledge and agency” should not assume the discrete nature of machines and humans with some sort of pre-planned boundary. Rather the boundary between humans and machines materialises during the encounter, and will vary based on the participants, and the setting. She described how we can learn from interactions during encounters, and how the boundaries between humans and machines systems can change over time and not be distinct. This ambiguity of interpretation is an avenue for exploration, not avoidance.
When we consider more than one-to-one encounters, the one-to-many and many-to-many, the framing needs a wider viewpoint (Suchman, 2006) that places less emphasis on individual transactions (regardless whether by machines or by humans) because agency resides “neither in us nor in our artifacts but in our intra-actions”. These interactions vary session-to-session, and group-to-group. The combined group agency is not entirely pre-scripted and predictable. It is situated action influenced by our experiences, conventions, traditions, knowledge, collaborations, and of course the plan.
From Contextual to Societal Co-Production
The move from HCI in the laboratory, to fieldwork, and now to the civic (Boehner and DiSalvo, 2016, and Vlachokyriakos at al, 2016) embedded in communities (Wright and Olivier, 2017), shifts the research role from being remote expert to working with and for individuals and communities. It is a step beyond the change form “user” to “social” approaches. It is a move to a broader societal approach, because the issues relevant to the people involved are about their lives – their needs, their concerns and their aspirations. These are inherently linked with service provision by the state, by markets, and by the third sector.
The discrete devices dreamt of in ubiquitous computing and the multi-located data everywhere fail to envision this inter-relational creation of agency, nor the ever-changing boundaries of a multitude of people with a multitude of machines, nor their potential multiple interpretations of their computer technology encounters (Sengers and Gaver, 2006). Like the various perspectives on personal data, different groups and individuals and society itself perceive different imprecise and shifting boundaries from an initial imbalance of understanding/knowledge as “agency emerges through interpretation” (Suchman, 2006) where the result is not pre-determined.
As researchers immersing ourselves, we become part of this fluid interconnected system, where problems and issues move into and out of focus at the same time as network connections and dependencies change, and where opportunities to use digital technologies may arise. The capacity for action is “relational, dynamic and collective” (Aanestad, 2003).
We need neither privileged experts nor privileged machines. The interdisciplinarity of HCI fits well with a multifaceted human society. Without defined privilege there is greater scope for inclusion of a wide range of beliefs, theories and ideas. Using co-production techniques, the plan and the action can emerge from a more ambiguous starting point.
Suchman proposed that machines and humans are complementary rather than equivalent. The outcome is not a fixed pre-determined outcome, but like Suchman’s flexible and moving “boundaries”, instead there is something which is relational, situational and changes over time. The outcome of an interaction cannot simply be pre-planned, but needs to consider the context, and the interactions between all the participating parties at the time. Similarly in our research with people (Wright and Olivier, 2017), the “problem” and the “solution” cannot be pre-planned. We do not have the solutions defined, nor the problems defined.
Co-production in encounters, and in design of systems, provides greater agency to individuals. Rather than being part of someone else’s opaque system, people can contribute to the creation their own systems where the social justice related properties of fairness, transparency, openness are also considered in the wider picture. This requires investment in local communities “building trust and, critically, distributing power”.
Full Spectrum HCI
Working as an applied engineer of sorts for most of my working life, much of my progression has been through gathering and applying knowledge as facts, in essence knowledge that can be learnt and therefore tested (Sfard, 1998). Engineering, with its emphasis on the “genius” role, likes to build bodies of knowledge around generalised methods. In contrast, in HCI we need to allow for, even promote ambiguity, a topic of surprise to me when it appeared early in our HCI for Digital Civics course. Solutions can arise from the participants, with their ideas, thoughts, skills and digital machines.
Various HCI approaches, visions and viewpoints provide alternative insights into issues. In a similar way, Digital Civics is an unconventional part of the HCI family – somewhat alternative and willing to challenge norms. It is not important exactly what Digital Civics is, but much more how it does it and the societal outcomes. Pushing the viewpoint, scope and framing wider to society allows us to consider socio-technical systems that go well beyond the absolute limits of machine computability identified by Turing (1936).
Returning to the earliest person in this story, Lovelace, the “first programmer” published her work during first-wave feminism (Bardzell, 2010). Suchman also suggested there is possibly much to learn from alternative theories such as found in art, and in feminist approaches (Rode, 2011), including the role of gender, equity, diversity, social justice, care and art.
Some characters in the historical references here have been towards the edges of society – outliers, their lives and motivations somewhat ambiguous. Society is comprised of individuals, and they are not uniform. There is great variety. It is comprised of unique individuals. With uniqueness comes a wide spectrum of ideas, thoughts, contributions. We should be inspired to look at the fringes, the specialist, the alternative niches in HCI and wider academia, to find novel views and ideas which might fit with this distributed local service provision.
Distributed Co-Produced Socio-Technical Systems
Suchman stated the important question is how to configure assemblages where humans and machines intra-act “responsibly and generatively” through acts of engagement. There is ambiguity here too (Gaver et al. 2003) in “the problem” and “the solution” which require people to participate in shared meaning making (Mikalsen et al, 2018). This is not ambiguity in the sense of speculative design (Wong and Khovanskaya, 2018) but ambiguity in the need, the process, the techniques, and the outcomes.
While it is attractive to undertake future research on the shiny new, there are problems of interest in what might seem the mundane and the ordinary. Innovation is not just invention – it can also be found in maintenance and care. People’s everyday lives. As Manzini said (2015a) we should be enabling people rather than services. These are not quite either “issue-based” or “ideologically based” publics (Le Dantec, 2016), but rather publics based on a widespread commonality. Possibly fundamentals such as accommodation, food, clothing and sanitation, but ones which cross social divides due to common needs. Not everyone is a parent; not everyone needs care; not everyone is unemployed.
So-called simple problems should be a target area of research interest, rather than the more glamourous complex problems. Research into these sometimes ignored or forgotten issues may identify commonalities that cross social, political and cultural boundaries (distributed issues) which could be fertile ground for social innovation (distributed solutions). They might reveal interesting hidden entanglements which are old or new, but just not shiny.
Distributed implies a wide network of individuals augmented by digital systems, forming a cohesive socio-technical public. The agency is diffusely spread throughout, built upon their intra-actions. It is not a uniform consistent mesh, but rather a flexible network of humans and technology, which has in-built resilience, varying granularities and structures, dynamic interconnections, and which can “handle change and stability simultaneously” (Duit et al, 2010).
This is not big HCI (Rogers, 2012) – instead it is about embracing the radical, and co-designing social cohesion through a distributed approach. Like the boundaries between humans and computers, the boundaries between researchers and “participants” need to be situational, contextual and allow for changes over time. People are also part of infrastructure. As a starting point we may want to consider how local council services might be provided through a distributed, rather than centralised approach, as well as being relational rather than transactional. Such work could “move from small-scale experimentation to solutions that are available to all” (Cottam, 2018).
Early theoretical understandings of the absolute limits of computability and the relevance of the human participant in socio-technical systems have come to the fore again in situated social perspectives. Keeping an open mind, allowing for a wide range of contributions from a diverse group of participants – researchers and non-researchers – simply reflects society. Interdisciplinarity is a benefit in HCI, so the multi-complexity of society should also be embraced in Digital Civics. We should consider the non-shiny and the simple, and what we might have missed in the rearview mirror, because the easily ignored may mean it has much wider applicability.
Future research should identify publics of commonality, the matters of everyday life we often take for granted, and empower those to undertake change that redistributes and develops a citizen economy. These may uncover solutions that are more sustainable and replicable. Issues that might seem to be simple could in fact shed light on social innovation for seemingly more wicked problems (Wright and Olivier, 2017).
Manzini’s emphasis on what design is, and who can do it, (2015b) is central to our thinking about social innovation, and building a sustainable and resilient culture. It was fitting that Shackel’s paper, considered to be the first in the field of HCI, was published in the journal called “Design”.
Photographs: Author’s own. The socio-technical (left to right): pedestrians on a street in Oxford, United Kingdom; view of Bogotá, Colombia; people shopping at a market in Barjac, France.
Margunn Aanestad. 2003. The camera as an actor: Design-in-use of telemedicine infrastructure in surgery. Computer Supported Cooperative Work.
Shaowen Bardzell. 2010. Feminist HCI : Taking Stock and Outlining an Agenda for Design. In SIGCHI Conference on Human Factors in Computing Systems (CHI’10).
Kirsten Boehner, Carl DiSalvo. 2016. Data, Design and Civics: An Exploratory Study of Civic Tech. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems.
John M Carroll. 2008-2018. Series Editor. Introduction: Synthesis Lectures on Human-Centered Informatics. Morgan & Claypool.
John M Carroll. 2003. Introduction: Toward a multidisciplinary science of human-computer interaction. In Carroll, J M, Ed., HCI Models, Theories and Frameworks. Morgan Kaufmann, San Francisco, CA.
Hilary Cottam. 2018. Radical Help: How we can remake the relationships between us and revolutionise the welfare state. Little, Brown Book Group.
Christopher A Le Dantec. 2016. Designing Publics (Design Thinking, Design Theory). The MIT Press.
Andreas Duit, Victor Galaz, Katarina Eckerberg, Jonas Ebbesson. 2010. Governance, complexity, and resilience. Global Environmental Change.
Douglas C Engelbart. 1962. Augmenting Human Intellect: A Conceptual Framework.
John Fuegi, Jo Francis. 2015. Lovelace & Babbage and the creation of the 1843 ‘Notes’. Ada User Journal.
William W Gaver, Jacob Beaver, Steve Benford. 2003. Ambiguity as a resource for design. In Proceedings of the conference on Human factors in computing systems – CHI ’03.
Jonathan Grudin. 2012. A moving target: The evolution of human-computer interaction. The Human-Computer Interaction Handbook–Fundamentals, Evolving Technologies, and Emerging Application.
Steve Harrison, Deborah Tatar, and Phoebe Sengers. 2007. The three paradigms of HCI. In the Conference on Human Factors in Computing Systems (CHI2007).
Andrew Hodges. 1983. Alan Turing: The Enigma. Burnett Books with Hutchinson.
Ada Lovelace. 1843. Translator’s notes to Sketch of The Analytical Engine Invented by Charles Babbage by L. F. MENABREA of Turin, Officer of the Military Engineers from the Bibliothèque Universelle de Genève, October, 1842, No. 82 With notes upon the Memoir by the Translator Ada Augusta, Countess of Lovelace. Retrieved 07 Dec 2018.
Ezio Manzini. 2015a. Design, When Everybody Designs. Medea Talk, 10 Sep 2015, YouTube retrieved 18 Nov 2018.
Ezio Manzini. 2015b. Design, When Everybody Designs: An Introduction to Design for Social Innovation. Translated by Rachel Coad.
Marius Mikalsen, Babak A Farshchian, Yngve Dahl. 2018. Infrastructuring as Ambiguous Repair: A Case Study of a Surveillance Infrastructure Project. Computer Supported Cooperative Work: CSCW: An International Journal.
Theodor H Nelson. 1960. Vision of Interconnectedness.
Antti Oulasvirta, Kasper Hornbæk. 2016. HCI Research as Problem-Solving. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems – CHI ’16.
Jennifer A Rode. 2011. A theoretical agenda for feminist HCI. Interacting with Computers.
Yvonne Rogers. 2012. HCI Theory: Classical, Modern, and Contemporary. Synthesis Lectures on Human-Centered Informatics.
Phoebe Sengers, Bill Gaver. 2006. Staying Open to Interpretation: Engaging Multiple Meanings in Design and Evaluation. In In Proc.of DIS’06.
A Sfard. 1998. On Two Metaphors for Learning and the Dangers of Choosing Just One. Educational Researcher.
Brian Shackel. 1959. Ergonomics for a computer. Design 120: 36-39.
Ben Shneiderman. 2011. Claiming success, charting the future: micro-HCI and macro-HCI. Interactions.
Lucy A Suchman. 1985. Plans and Situated Actions: The Problem of Human Machine Communication.
Lucy Suchman. 2006. Human-machine reconfigurations: Plans and situated actions, 2nd edition.
Alan M Turing. 1936. On Computable Numbers, with an Application to the Entscheidungsproblem. Mathematica.
Alan M Turing. 1948. Intelligent Machinery.
Vasillis Vlachokyriakos, Clara Crivellaro, Christopher A Le Dantec, Eric Gordon, Pete Wright, Patrick Olivier. 2016. Digital Civics: Citizen Empowerment With and Through Technology. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems – CHI EA ’16.
Richmond Y Wong, Vera Khovanskaya. 2018. Speculative Design in HCI: From Corporate Imaginations to Critical Orientations Richmond. In New Directions in Third Wave Human-Computer Interaction: Volume 2 – Methodologies.
Peter Wright, Patrick Olivier. 2017. Digital Civics and the Cities Challenge. Open Lab, Newcastle University.
Author’s own. Close-up from “Turing IV – Machinery”, Turing Suite, Eduardo Paolozzi , 2000, colour silkscreen print, hors commerce originally given to Hodges by Paolozzi. Incorporating excerpts from Hodges 1983 and Turing 1948.
Verbatim from each source cited.
Author: Colin Watson