What began with research on the interactions of individuals with computer technology, transformed to the social, and now also society. From the precision of one, to the ambiguity of the many, from a universal machine to a universe of possibilities.
Early computing pioneers were very much aware of computational limits, recognising the pivotal role humans have in the overall system. With an innate human preference for shiny new things, initially research in Computer Science and later Human Computer Interaction (HCI) understandably concentrated efforts on the machine side of these systems, rather than the human and social aspects.
The move to the social, especially Suchman’s work on Plans and Situated Actions saw a shift in emphasis, with greater influence being assigned to the human side and the context of the interaction. The seeds of co-production appear in Suchman’s work contributing to our research including the turn to the civic where it is recognised citizens can collectively contribute to social justice, including in the provision of local services.
The interdisciplinarity of HCI is a natural fit with the multi-complexity of society – it is not a homogenous mix. HCI research has benefitted from looking to and connecting with the alternative, the niche, the outliers. The socio-technical cannot be defined with precision; we cannot assume what problems need to be solved, nor what the solutions will be.
Future research should keep as many avenues open as possible, and seek outliers and oddities. Our research should not attempt to follow well-worn paths nor focus on the shiny new. It is suggested that examining the mundane, the ordinary and the simple may provide worthwhile insights – big things have small beginnings.
Photographs: Author’s own taken at The National Museum of Computing, Bletchley Park. The electro-mechanical (left to right): reconstructed Turing-Welchman Bombe machine wiring; Tony Sale with his rebuilt Mk 2 Colossus; reconstructed Turing-Welchman Bombe machine drums.
Multiplicity of Perspectives
The foundations of HCI are not that much younger than the foundations of Computer Science itself. The interdisciplinarity of HCI mirrors the story of people central to computing’s journey, many of whom were unconventional, alternative, outsiders and somewhat controversial within the societal norms they lived in. This acceptance of the unusual and different perspectives has helped drive innovation in HCI, and in turn this can be used to foster innovation in society.
My own first encounters with computers were learning to programme at school aged 13, involving using a sharp pencil to write BASIC code on programming sheets, which were sent to the University of Edinburgh for keying onto punch cards and submission of those to “the computer”. The printed output a week later was disappointedly often “Syntax error at line 10”.
From those remote interaction, the field of HCI has been transformed. HCI has broken out of “disciplinary boundaries” (Shneiderman, 2011) to become an interdisciplinary area with a dynamic and fluid mix of researchers with backgrounds in computer science, information studies, psychology, sociology, political science and art. It is like design which Manzini (2015a) says is a “third way of doing things between hard science and human science”. Carroll (2008-2018) writes HCI “is the intersection of the cultural, the social, the cognitive, and the aesthetic with computing and information technology”.
So how did HCI get to where we are now?
In the Beginning
We have come a long way. Charles Babbage’s interaction with computational machines was very much physical with the construction of the analogue machines using mechanical parts. His colleague Ada Lovelace published theories in a scientific journal as notes to a translation (Lovelace, 1843) which are now recognised as the first computer algorithms. Lovelace’s insights may not have been possible without a broad interdisciplinary background. She was a mathematician, an engineer, a writer, an illustrator, and a computer scientist. Lovelace’s contributions were largely overlooked until more recently, but opinion is that she had greater insight than Babbage and “went beyond number-crunching to see possibilities for wider applications” (Füegi and Francis, 2015). Such ingenuity and foresight.
Babbage might be known as the “father of computing”, but it is Alan Turing who is recognised as the “father of the modern computer” (Hodges, 1983). Wartime efforts spurred on developments in computing. Turing had little concern for status or the class and privilege structures in post-war Britain (Hodges, 1983). With a broad background in in science, maths and philosophy, and his research work’s considerable legacy included the concept of a universal machine, and also the absolute limits of the computable.
In those early years of computing, the interface between humans and computers was changing from electronic rewiring, to printed or punched paper, where interaction was limited to programming, operating the hardware and utilising the output (Grudin, 2012).
Christopher Strachey founded the Programming Research Group at the University of Oxford, and was the university’s first Professor of Computer Science. This is where I later studied a computer science conversion course, in the early days of the public internet. Strachey, another outsider at the periphery of traditional society, was invited to the University of Manchester by his friend Turing. True multimedia output may first have occurred in 1951 when Strachey programmed a draughts game which played the national anthem on termination. Later Strachey developed a program to read and play any tune, and some recordings from that year were recently restored and are the earliest known recordings of computer-generated music.
As a friend, Hodges invited me to his 2016 Strachey Lecture – The Once and Future Turing, where I was intrigued to see the inclusion of a photo from the Turing Archive of a printed computer-printed personal message annotated and sent by Turing to his associate and close friend Robin Gandy, demonstrating early “consumer computing”. During the same lecture, Hodges explains that in the discussion of the decision problem (Turing, 1936), Turing “invented the mentality of computer programming” from the rigour of mathematical foundations, and that he was not thinking just about the mind of the programmer, but an analysis “of the action of the human mind and what it is a person is doing when they are computing”.
Early formative steps in HCI came from the fields of research into human engineering, human factors and ergonomics, where the emphasis was on efficiency of action. What is seen as one of the first paper in the HCI field, but well before the term “HCI” was introduced, Shackel (1959) describes how ergonomic design principles can be used to reduce mistakes when “[re-programming] from one problem to solve another”.
With the development of vacuum-tube displays, screens provided greater interaction flexibility, and the humans interacting with the computers expanded beyond mathematicians and technicians. As we shall see later, this expansion of the “user base” was, and continues to be one of the main driving factors in the changing scope of HCI research.
From One to Many
Even in the early years, visionaries like Nelson (1960), foresaw the vast inter-connectedness we have today and Engelbart contemplated human augmentation (1962) by machines. In 1965 it was Nelson who introduced the term hypertext, and twenty years later I became a member of the Edinburgh Home Computer Club and first got online, joining Micronet 800 and other services on BT’s Prestel videotext service. It was another ten years before I created my first webpage, something which afterwards turned into a commercial career.
What began primarily as user interaction design, HCI has changed, morphed, and reinvented itself many times. Invigorated by its interdisciplinarity, the field comprises an ever-shifting cornucopia of viewpoints and visions. From a human factors/engineering approach, this shifted to a cognitive approach drawn from psychology, then to alternative cognitive approaches, more social approaches including situated action, ethnomethodology and ethnography, and then drawing theory from a wider range of fields, including activity theory and grounded theory (Rogers, 2012, and Harrison et al, 2007).
These viewpoints and visions have also reflected the changing role of the “user”. Initially firmly rooted in the consideration of human factors and ergonomics from an operator’s point of view, this changed as the outputs of the machines were driven by economic needs to management information systems, where the understanding of typically printed output became a more prevalent driver. With a growing number of programmers, the field encompassed the area of how software in constructed (the user as a programmer). Cognitive and design approaches developed, drawing on experience from other fields, to inform how software is developed (Carroll, 2003).
Driven by the vast expansion of computing in the 1980s into workplaces, the home and the mass consumerisation in the 1990s and 2000s, there was an increase in ideas drawn from social sciences, with less emphasis on traditional cognitive approaches.
Additionally, the scale of “user” considered in HCI has changed. HCI originally focused on individual interactions, but this has expanded to include groups of people, and more recently to vast numbers of people (through ideas like ubiquitous computing, social computing and the growth of the data society). In a constructivist approach both the past (knowledge) and current situation impinge on our response/interactions and societal-scale groupings can multiply this effect.
But first, consider one theory from the late 1980s which examined how the context of the interaction between humans and technologies leads to intelligent action. Suchman (2006) wrote in a discussion of critiques of the term “user” which “singularizes what is actually a multiplicity and fails to differentiate actors with very different relations to a given artifact”.
The Social with Plans and Situated Actions
Suchman (1985, 2006) touched on a topic close to Turing’s consideration of “what a person is doing when they are computing” (1936) which resonates in his later work on intelligent machines (1948).
Suchman studied photocopier users as they try to assist each other. My own understanding of Plans and Situated Actions (P&SA) was greatly helped by the description of Turnbull’s examination of the debate “around the presence (or absence) of plans in the building of the great medieval Gothic cathedrals” (1999). Turnbull proposed cathedrals were created by collaborative encounters as the result of iterative interactions between people, practices and materials relying on the essential components of “talk, tradition and templates”. Since the middle ages, there has been an increasingly centralised “genius” role in the construction of buildings – the architect and plan. This is common in other fields, for example, conventional western medical interventions remove humans of their own agency (Thompson, 2005).
Suchman challenged the genius role of plans in human-computer systems highlighting that the “attribution of knowledge and agency” should not assume the discrete nature of machines and humans with some sort of pre-planned boundary. Rather the boundary between humans and machines materialises during the encounter, and will vary based on the participants, and the setting. She described how we can learn from interactions during encounters, and how the boundaries between humans and machines systems can change over time and not be distinct. This ambiguity of interpretation is an avenue for exploration, not avoidance.
When we consider more than one-to-one encounters, the one-to-many and many-to-many, the framing needs a wider viewpoint (Suchman, 2006) that places less emphasis on individual transactions (regardless whether by machines or by humans) but that agency resides “neither in us nor in our artifacts but in our intra-actions”. These interactions vary session-to-session, and group-to-group. The combined group agency is not entirely pre-scripted and predictable. It is situated action influenced by our experiences, conventions, traditions, knowledge, collaborations, and of course the plan.
From Contextual to Societal Co-Production
The move from HCI in the laboratory, to fieldwork, and now to the civic (Boehner and DiSalvo, 2016, and Vlachokyriakos at al, 2016) embedded in communities (Wright and Olivier, 2017), shifts the research role from being remote expert to working with individuals and communities. It is a step beyond the change form “user” to “social” approaches. It is a move to a broader societal approach, because the issues relevant to the people involved are about their lives – their needs, their concerns and their aspirations. These are inherently linked with service provision by the state, by markets, and by the third sector.
Discrete devices dreamt of in ubiquitous computing and the multi-located data everywhere fail to envision this inter-relational creation of agency, nor the ever-changing boundaries of a multitude of people with a multitude of machines, nor their potential multiple interpretations of their computer technology encounters (Sengers and Gaver, 2006). Like the different perspectives on personal data, different groups and individuals and society itself perceive different imprecise and shifting boundaries as “agency emerges through interpretation” and the result is not pre-determined.
As researchers immersing ourselves, we become part of this fluid interconnected system, where problems and issues move into and out of focus at the same time as network connections and dependencies change, where opportunities to use digital technologies may arise. The capacity for action is “relational, dynamic and collective” (Aanestad, 2003). Not just “civic technology”, not transactional. More embedded, relational, and innovative.
We neither need privileged experts nor privileged machines. The interdisciplinarity of HCI matches well with the multifaceted human society. There is scope for inclusion of a wide range of beliefs, theories and ideas. Suchman wrote that where there is an initial imbalance of understanding/knowledge, agency emerges during interpretation. Using co-production techniques, the plan and the action can emerge from a more ambiguous starting point.
Suchman proposed that machines and humans are complementary rather than equivalent. The outcome is not a fixed pre-determined outcome, but like Suchman’s flexible and moving “boundaries”, instead is something which is relational, situational and changes over time. The outcome of an interaction cannot simply be pre-planned, but needs to consider the context, and the interactions between all the participating parties at the time. Similarly in our research with people (Wright and Olivier, 2017), the “problem” and the “solution” cannot be pre-planned. We do not have the solutions defined, nor the problems defined.
We do not need yet more expert-driven top-down technology-focused initiatives. This co-production of systems also provides greater agency to individuals. Rather than being part of someone else’s opaque system, people can contribute to the creation their own systems where the social justice related properties of fairness, transparency, openness are also considered in the wider picture. This requires investment in local communities “building trust and – critically – distributing power”.
Full Spectrum HCI
Working as an applied engineer of sorts for most of my working life, much of my progression has been through gathering and applying knowledge as facts, in essence knowledge that can be learnt and therefore tested (Sfard, 1998). Engineering likes to build bodies of knowledge around the generalised methods. In contrast, in HCI, we need to allow for, even promote ambiguity, a topic of surprise when it appeared early in our HCI for Digital Civics course. The solutions should arise from the participants, with their ideas, thoughts, skills and digital machines.
Various HCI approaches, visions and viewpoints provide alternative insights into issues. In a similar way, Digital Civics is an unconventional part of the HCI family – somewhat alternative and willing to challenge the norm. It is not important exactly what Digital Civics is, but much more how it does it and the societal outcomes. Pushing the viewpoint, the scope and framing, wider to society allows us to consider socio-technical systems go beyond the absolute limits of computability identified by Turing (1936).
Returning to another of the earlier characters in this story, Lovelace, the “first programmer” published her work during the first-wave feminism phase (Bardzell, 2010). Indeed, Suchman suggests there is possibly much to learn from alternative theories such as found in art, and in feminist approaches (Rode, 2011), including the role of gender, equity, diversity and social justice, and art.
Some characters in the historical references have been towards the edges of society, outliers, their lives and motivations somewhat ambiguous. Society is comprised of individuals, and they are not uniform. There is great variety. It is comprised of individuals. Individuals are unique. With uniqueness comes a broad spectrum of ideas, thoughts, contributions. We should be inspired to look at the fringes, the specialist, the alternative niches in HCI and wider academia, to find novel views and ideas which might fit with this distributed local service provision.
Distributed Co-Produced Socio-Technical Systems
Suchman stated the important question is how to configure assemblages where humans and machines intra-act “responsibly and generatively” through acts of engagement. There is ambiguity here too (Gaver et al. 2003) in “the problem” and “the solution” which require people to participate in shared meaning making (Mikalsen et al, 2018). This is not ambiguity in the sense of speculative design (Wong and Khovanskaya, 2018) but ambiguity in the need, the process, the techniques, and the outcomes.
While it is attractive to undertake future research on the shiny new, there are problems of interest in what might seem the mundane and the ordinary. People’s everyday lives. As Manzini said (2015a) we should be enabling people rather than services. Like the boundaries between humans and computers, the boundaries between researchers and “participants” need to be situational, contextual and allow for changes over time.
Research on these issues might uncover solutions that are more replicable and thus sustainable. I have a suspicion that so-called simple problems might be a target area of interest, rather than the more glamourous complex problems. These simple problems might reveal interesting hidden entanglements which are new, just not shiny. Research into some of these ignored or forgotten issues may be able to identify commonalities that cross social, political and cultural boundaries (distributed issues) which could be fertile ground for social innovation (distributed solutions).
By “distributed” in this context I mean a wide network of individuals augmented by digital systems, forming a cohesive socio-technical public. These are not quite either “issue-based” or “ideologically based” publics with multiple attachments which are “distributed networks of people and artifacts, communities and institutions, and instruments of authority and power” (Le Dantec, 2016), but rather publics based on a commonality.
It is distributed because the agency is diffusely spread throughout, built upon the intra-actions. This is not a uniform consistent mesh, but rather a flexible network of humans and technology, which has in-built resilience, varying granularities and structures, and interconnections that changes over time.
This is not big HCI (Rogers, 2012), instead it is about embracing the radical, and co-designing social cohesion through a distributed approach. Remember, people are also infrastructure. As a starting point we may want to consider how local council services might be provided through a distributed, rather than centralised approach, as well as being relational and not transactional. Such work could “move from small-scale experimentation to solutions that are available to all” (Cottam, 2018).
When considering the societal viewpoint for the impact of our work, where we have even less confidence of whether “research is valid or follows the right approach” (Oulasvirta and Hornbæk. 2016), the measure of our work in local services should be based on societal impact, and not primarily on measures of academic productivity, contribution to theory or indeed cost reduction or market-based efficiencies.
Early theoretical understandings of the absolute limits of computability and the relevance of the human participant in socio-technical systems have come to the fore again in situated social perspectives. Keeping an open mind, allowing for a wide range of contributions, from a diverse group of participants – researchers and non-researchers – simply reflects society. Interdisciplinarity is a benefit in HCI, so the multi-complexity of society should also be embraced in Digital Civics. We should consider the non-shiny and the simple, as the easily ignored may mean it has much wider applicability.
Future research should identify publics of commonality, the matters of everyday life we often take for granted, and empower those to undertake change that redistributes and develops a citizen economy. These may uncover solutions that are more sustainable and replicable. Issues that might seem to be simple could in fact shed light on social innovation for wicked problems (Wright and Olivier, 2017).
Manzini’s emphasis on what design is and who can do it (2015b) is central to our thinking about social innovation, and building a sustainable and resilient culture. It was fitting that Shackel’s paper, considered to be the first in the field of HCI, was published in the journal called “Design”.
Photographs: Author’s own. The socio-technical (left to right): pedestrians on a street in Oxford, United Kingdom; view of Bogotá, Colombia; people shopping at a market in Barjac, France.
Margunn Aanestad. 2003. The camera as an actor: Design-in-use of telemedicine infrastructure in surgery. Computer Supported Cooperative Work.
Shaowen Bardzell. 2010. Feminist HCI : Taking Stock and Outlining an Agenda for Design. In SIGCHI Conference on Human Factors in Computing Systems (CHI’10).
Kirsten Boehner, Carl DiSalvo. 2016. Data, Design and Civics: An Exploratory Study of Civic Tech. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems.
John M Carroll. 2008-2018. Series Editor. Introduction: Synthesis Lectures on Human-Centered Informatics. Morgan & Claypool.
John M Carroll. 2003. Introduction: Toward a multidisciplinary science of human-computer interaction. In Carroll, J M, Ed., HCI Models, Theories and Frameworks. Morgan Kaufmann, San Francisco, CA.
Hilary Cottam. 2018. Radical Help: How we can remake the relationships between us and revolutionise the welfare state. Little, Brown Book Group.
Christopher A Le Dantec. 2016. Designing Publics (Design Thinking, Design Theory). The MIT Press.
Douglas C Engelbart. 1962. Augmenting Human Intellect: A Conceptual Framework.
John Fuegi, Jo Francis. 2015. Lovelace & Babbage and the creation of the 1843 ‘Notes’. Ada User Journal.
William W Gaver, Jacob Beaver, Steve Benford. 2003. Ambiguity as a resource for design. In Proceedings of the conference on Human factors in computing systems – CHI ’03.
Jonathan Grudin. 2012. A moving target: The evolution of human-computer interaction. The Human-Computer Interaction Handbook–Fundamentals, Evolving Technologies, and Emerging Application.
Steve Harrison, Deborah Tatar, and Phoebe Sengers. 2007. The three paradigms of HCI. In the Conference on Human Factors in Computing Systems (CHI2007).
Andrew Hodges. 1983. Alan Turing: The Enigma. Burnett Books with Hutchinson.
Ada Lovelace. 1843. Translator’s notes to Sketch of The Analytical Engine Invented by Charles Babbage by L. F. MENABREA of Turin, Officer of the Military Engineers from the Bibliothèque Universelle de Genève, October, 1842, No. 82 With notes upon the Memoir by the Translator Ada Augusta, Countess of Lovelace. Retrieved 07 Dec 2018.
Ezio Manzini. 2015a. Design, When Everybody Designs. Medea Talk, 10 Sep 2015, YouTube retrieved 18 Nov 2018.
Ezio Manzini. 2015b. Design, When Everybody Designs: An Introduction to Design for Social Innovation. Translated by Rachel Coad.
Marius Mikalsen, Babak A Farshchian, Yngve Dahl. 2018. Infrastructuring as Ambiguous Repair: A Case Study of a Surveillance Infrastructure Project. Computer Supported Cooperative Work: CSCW: An International Journal.
Theodor H Nelson. 1960. Vision of Interconnectedness.
Antti Oulasvirta, Kasper Hornbæk. 2016. HCI Research as Problem-Solving. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems – CHI ’16.
Jennifer A Rode. 2011. A theoretical agenda for feminist HCI. Interacting with Computers.
Yvonne Rogers. 2012. HCI Theory: Classical, Modern, and Contemporary. Synthesis Lectures on Human-Centered Informatics.
Phoebe Sengers, Bill Gaver. 2006. Staying Open to Interpretation: Engaging Multiple Meanings in Design and Evaluation. In In Proc.of DIS’06.
A Sfard. 1998. On Two Metaphors for Learning and the Dangers of Choosing Just One. Educational Researcher.
Brian Shackel. 1959. Ergonomics for a computer. Design 120: 36-39.
Ben Shneiderman. 2011. Claiming success, charting the future: micro-HCI and macro-HCI. Interactions.
Lucy A Suchman. 1985. Plans and Situated Actions: The Problem of Human Machine Communication.
Lucy Suchman. 2006. Human-machine reconfigurations: Plans and situated actions, 2nd edition. https://doi.org/10.1017/CBO9780511808418
Charis Thompson. 2005. Making Parents: The Ontological Choreography of Reproductive Technologies. In Making Parents: The Ontological Choreography of Reproductive Technologies.
David Turnbull. 1999. Masons, Tricksters, and Cartographers: Comparative Studies in the Sociology of Scientific and Indigenous Knowledge. London: Harwood Academic Publishers.
Alan M Turing. 1936. On Computable Numbers, with an Application to the Entscheidungsproblem. Mathematica.
Alan M Turing. 1948. Intelligent Machinery.
Vasillis Vlachokyriakos, Clara Crivellaro, Christopher A Le Dantec, Eric Gordon, Pete Wright, Patrick Olivier. 2016. Digital Civics: Citizen Empowerment With and Through Technology. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems – CHI EA ’16.
Richmond Y Wong, Vera Khovanskaya. 2018. Speculative Design in HCI: From Corporate Imaginations to Critical Orientations Richmond. In New Directions in Third Wave Human-Computer Interaction: Volume 2 – Methodologies.
Peter Wright, Patrick Olivier. 2017. Digital Civics and the Cities Challenge. Open lab, Newcastle University.
Author’s own. Close-up from “Turing IV – Machinery”, Turing Suite, Eduardo Paolozzi , 2000, colour silkscreen print, hors commerce originally given to Hodges by Paolozzi. Incorporating excerpts from Hodges 1983 and Turing 1948.
Verbatim from each source cited.