From Human – Machine interaction to Smart Citizens’ Choice

I can still recall to my mind the confusion created in my brain in 2010, when our professor during a discussion described to us the procedure that he had to do as a Computer Science student in Greece before the middle 80s every time that he had to submit a programming assignment. His words were describing exactly what I had studied as the first steps of Computer Programming history, which back then was a part of a distant past. Although, when one Professor almost in the age of my parents was describing to us that “We had to visit the central sciences building of Aristotle University of Thessaloniki during the morning, as there, was the only available Computer, take a number and wait for the time that the Computer will be available for us. When after some hours we had some minutes to use the computer, we were using the already prepared by us punched cards [1], to compile our program. In case there was an error message, we should leave, make any appropriate changes and try again the following day”, the “distant past” became “Some years ago”. Thus, I realised it was the time for me to search more about the history of Human-Computer Interaction, as every invention of the sector was much closer than I was thinking.

The Past:

Even though, initially the idea was compared to dancing lessons for elephants, the great barrier of visiting buildings that host a Computer was ready to break forever, when IBM introduced the first personal computer in August 1981 [2]. The IBM PC was coming in an affordable cost with incredible for that time capabilities. At this point has to be mentioned that computers that could be handled only by one person were already available some years earlier [3], with limited capabilities, and were mostly available as kits, but the reason that the IBM PC is a milestone in the PC history is that it was the real progenitor of all the following models [4]. The IBM PC was based in MS-DOS operating system and it was accepting commands as written input.

Although, the real evolution in relation to technology and Human Computer Interaction have started exactly when, three separated by that time worlds (Academic, Corporate and Commercial) parallelized, as is described by Brad Myers [5]. Only after this synergy, the available systems were closer to what we know today as Personal Computer, with the coexistence of mouse, Graphical User Interface (GUI) and Operating System in one financially affordable machine. More or less, this is also the point of the history that the integration between the physical with the digital world started, by the representation of physical world objects in the GUI and the use of Natural Language commands in the shape of menus and buttons. In this point is clear that the absolute goal of HCI is to imitate the real world as a step of making the GUI more user friendly. This goal was firstly achieved by the company Apple Inc. and the creation of the first Macintosh which was introduced on January 1984 by Steve Jobs [6].

After almost two decades of Improvements of personal computers, the evolution of Smartphone technology brought into the light, again by the same company, but this time with the first IPhone. For one more time, we can notice that even though the IPhone was not the first smartphone in the market, is mentioned as a milestone, thus because it introduced features as the multi touch screen, which as a functionality was not available earlier. Possibly this means that no matter how great an invention is, the detail that can transform a product to a real milestone is the recognition and most importantly the adoption of it in the everyday life. The high adoption of basically the functionalities that the smartphone was carrying from the people, changed the whole perception till then about the portable computers. In a really short period the biggest part of the population in the advanced countries was carrying a portable computer, with capabilities higher even than the existing computers, just in their pocket. [7]

The use of the fingers instead of a mouse is even more direct and this fact brings the humans one step closer to the digital world. The use of sliding, swapping and tapping, instead of scrolling and “clicking” are able to simulate or make use of natural human gestures. Moreover, the combination of the internet with the smartphone technology is giving access to the computing technology anytime from literally anywhere!

Although the term “Privacy” is not term that was constructed for the needs of information technology and was an important issue from the ancient years already [29]. More or less this is the time that we started be more sceptical about our privacy, as our lives with the used of smartphones became more exposed than earlier.

Today:

As the means needed to construct technology (microchips, boards, batteries etc.) are becoming constantly smaller, the technology is becoming an integral part of everyday life and in some cases is even able to replace the physical world by bringing it in home. With the use of a variety of sensors the human contact and movement is able to be adapted in digital environments and vice versa. Humans are able to explore fictional or non-fictional worlds and interact with them, have the ability to measure their physical activity, be informed about their health status [8] and interact with objects remotely.
IBM PC in size of salt

In other words, the rapid evolution of technology has brought into our lives a huge amount and variety of computers. Some of them are visible (Laptops, Tablets, Smartphones), while other are integrated in electronic devices such as TVs, home electronic appliances, cars etc. or even in everyday objects (keys, toys, cards). By arriving in the present, the size of a computer device is becoming unbelievably small and thus, the introduction of Internet of Things which has as main target the interconnection of the contents of physical world (objects, sensors’ measurements, animals, humans) and the integration of it into the digital world. While at the time that this article is written we can even officially mention the existence of programmable materials [9].
[1] IBM’s “world’s smallest computer”

As we are still speaking about new technologies, for the time all these systems are mainly autonomous and they are working independently the one of the other, so we can possibly admit that we have on our hands technology for which we don’t know every potential use. In parallel all these conveniences made us be really used in having access to information and data, but in the same time we are not really conscious about the data which are producing ourselves with all these computers and sensors around us, but also we possibly have zero idea about the safety of them. As I underlined in an article earlier [10], my main belief is that science in general, or information technology specifically by itself is just a mean, a really useful tool that if is in good hands can possibly exemplify our life, or it can be turned to something really dangerous if is used by wrong hands.

But before a detailed discussion about the ethical aspects which could arise from the integration and interconnection between all these available data, let’s briefly examine some ways that we tend to make use of them in relation with the society now.

Digital Health:

The use of new technologies in health sector is the most of the times, the first topic that will be examined with the introduction of every new technology, as health is an important matter for everyone. In their paper [11] Yassine A. et.al are making a proposal of use of all the data that are currently produced from a smart home in order to analyse occupants’ everyday behaviour in order to identify declination of human daily routine tasks and predict instabilities related to occupants’ health and well being.  The occupant on that case is a passive subject who produces data which will be analysed in order to produce a result that will be possibly known both to public health services and to the occupant of the home. From the other side there is another proposal in which the human is an active subject who tweets or sends via text messages, measurements related to health in order to receive a real-time prediction about his/her current health status [12]. While other projects are mostly focused in increasing the speed and the effectiveness of healthcare system by offering real time retrieval mechanisms for the constructions of patients’ medical situation by combining the medical history, with proposed diagnosis from the experts and data retrieved by sensors, without the need of human intervention [13]. Even though, it is out of the scope of the current article, it has to be mentioned also the great research that is taking place for using new technologies for the treatment of serious diseases. Finally, it should be noted that even though the research in relation to the integration of new technologies with public health is in early stages in relation to the time, but not with the quality of the proposed projects and products, is important that the sector of the security and privacy of every application is working in parallel with every proposed advancement [14].

Digital Learning:

As the use of technology is established as a part of everyday life, the interdisciplinary research took advantage of it, to examine how technology can assist or even change the traditional ways of learning, going a step further than just the remote learning (e-learning). The wider the advancements in technology the more revolutionary are the ideas that are proposed in order to change the procedure of the traditional learning. Some examples of as mentioned revolutionary for the time ideas, could include projects from a proposal of shaping the learning procedure and constructing the teaching material by making use of Social Media [15], a technology that is in students’ daily routine; to the construction of an intelligent tutoring system based on gamification, which is using data extracted from motion sensors and the combination of them with linguistic notions to help elementary students with the better understanding and handling of the language [16]. Moreover, is worth to mention at this point that the rapid evolution of technology and the advances in data handling, opened the horizons for the creation of new sectors as the Citizen Science. In this section Citizen science is not examined from its core scope, which is the involvement of citizens in science and the decrease of the gap between the citizens and the research and academic societies. It is mostly mentioned here, as an alternative form of lifelong learning, which enhances civic participation and gives to every interested in science individual the opportunity to learn and be a part of a research topic without the need of any prior knowledge on the topic [17].

Digital Democracy:

The sector of Digital Democracy, which from a social scope is possibly the most revolutionary one has as main target the use of digital technologies to enhance social participation, fight social inequalities and transfer the power in a fair for the whole society levels. It is quite interesting, that the tries to use the existing technology to transform the society started quite early in internet age, by projects oriented in community enhancement with the creation of living laboratories, which were inviting the participation of citizens to achieve their projects’ scopes [18]. Fortunately, the topic of digital democracy is continuously using the technological advancements in order to achieve its’ targets and to find new ways to support society. This support is sometimes raising, in a structured and formal way as an outcome of a research project, which has as a target to enhance the citizen participation [19] and some other is created directly by the citizens in a more wild way, without the realisation that the action taking place is an innovative way of organisation [20].

Indeed, as the generation who grew up while extreme changes in technology have been occurred, we managed to find nice ways to make use of these advancements. Possibly some of us have never faced a difficult situation because of the way that we are using technology, but is this true for everyone?

Unfortunately not, as some people have already experienced quite difficult life experiences just because of a social media profile [21]. And as we can clearly see, the problem was not about the technology itself, but about the way in which people used it.  Although, as humans we tend to think that unlikely events are not going to happen to us, in my opinion this is not bad, is just human nature, and this is the reason we sometimes fail to understand the importance of taking quick action in order to prevent unwanted situations.

The Future:

As the digital world tend to stop looking so obvious, being constructed by computer devices which do not require the existence of a screen, as it was predicted by Weiser in 1991 [22], the interaction between the human and the computers is clearly getting a new shape. So as Ubiquitous Computing is not a part of the future [23] anymore, the issues which have to be resolved are quite a lot.

From a technical point of view there is a need for solutions in relation to the fact that everything will be connected with everything! How all the devices around us will be able to communicate between each other in order to serve the common target of making human life easier? [24] Or how a human will be able to communicate with a device without creating confusion to other smart objects connected to the network? For example, what is going to happen if a safety system will understand a gesture needed to be done by a human in order to lower the lights as an alarm or the opposite? Furtherly, there is a need of clarifying, how a smart environment should work when a lot of people are in the same place, while they are behaving normally, discussing and making gestures, but in the same time, they have different preferences, needs and targets. Undoubtedly, factors as the determination of which subject matters more for a smart system  [25] or the collaboration of all this technology for the common good are really important factors that scientific research has to resolve, but there are also other “parallel” issues of equal importance that should be taken into account the same time or even before.

As the computers are an integral part of our lives already and are going to take a greater part of our daily routine in the near future, the ethical aspects of every technological advancement should be deployed in almost the same time with every introduced technology, as our digital footprints [21], are going to be created mostly unconsciously. Possibly some of the following questions are looking funny, but who is going to be responsible to fix a smart home in the case of a general crash? And in case, a smart home or a smart city crash is an easily resolvable issue, will the team which will offer the solution, have access to the data produced by it all the other times of the day? Or what is going to happen if the data or the control of a smart environment will be available to wrong hands?

 China social credit system
Going a step further, on the frame of a smart city, in my point of view, it should be clear and safely assured, that the data collected by the citizens are used only to make their life easier, and the way that the citizens can have access to view and modify their data in real time [26]. Even if I don’t agree with the use of data produced by citizens for the sake of surveillance, I can respect the research done during Maurtvedt’s thesis in relation to the China’s social credit system [27] and understand the point of the final conclusion, which states that the most important issue is not the gathering but the usage of the data, as the gathering could end up in a real nightmare in case the data are in wrong hands. Although, I am not sure how ethical is the rewarding or downgrading of citizens for their social behaviour, based on data collected by millions of governmental cameras.

[2] China’s Social Credit System

In Conclusion, undoubtedly in the next few years we will face really big changes in the communication between humans and computers, for which is not really known the positive or the negative impact that they will have in our daily lives. As every other tool, the Information Technology can be used both in good or bad ways [10]. What we have to take really seriously as a generation, is that the impact that the use of these tools will have, will be very broad and important. For that reason, is really critical that the human not as just a factor, but as a real instance will be taken into account, not only with all the experiences [28], abilities or disabilities which humans have, but also with the rights and the freedom they deserve. The most common outcome of every discussion in relation to the tendency of modern technologies, is more or less that we will be almost forced to accept the norms of the new smart society proposed, without having the right to choose. So, I am choosing to conclude this post with the belief that technological advancements will be used in order to enhance smart citizens participation, who will be able to decide which tools (if any) they want to use and in which way, in order to  preserve their rights and make their lives better.

 

“Designing for humans, without having the humans in the centre of the design process, is like preparing sticky toffee pudding for  a diabetic person!”

 

References:

[1] Fisk, D. 2005. Programming with Punched Cards. [pdf] Available at: http://www.columbia.edu/cu/computinghistory/fisk.pdf [Accessed 10 Dec. 2018].

[2] The birth of the IBM PC. (n.d.). Available at: https://www.ibm.com/ibm/history/exhibits/pc25/pc25_birth.html [Accessed 10 Dec. 2018].

[3] Paul Freiberger and Michael Swaine. 1984. Fire in the Valley: the Making of the Personal Computer. McGraw-Hill, Inc., New York, NY, USA.

[4] Computers | Timeline of Computer History. (n.d.). Available at: http://www.computerhistory.org/timeline/computers/ [Accessed 10 Dec. 2018].

[5] Brad A. Myers. 1998. A brief history of human-computer interaction technology. interactions 5, 2 (March 1998), 44-54. DOI=http://dx.doi.org/10.1145/274430.274436

[6] Barney, J. B. (1995). Looking inside for Competitive Advantage. Academy of Management Executive, 9, 49-61.

[7] X. Li et al., “Smartphone Evolution and Reuse: Establishing a More Sustainable Model,” 2010 39th International Conference on Parallel Processing Workshops, San Diego, CA, 2010, pp. 476-484. doi: 10.1109/ICPPW.2010.70 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5599108&isnumber=5599082

[8] Crews, Ryan & King, Abagayle L. & Yalla, Sai Vikas & Rosenblatt, Noah. (2018). Recent Advances and Future Opportunities to Address Challenges in Offloading Diabetic Feet: A Mini-Review. Gerontology. 64. 10.1159/000486392.

[9] Centre, T. B. (2017, February 02). 4D printed programmable materials. Available at: https://www.youtube.com/watch?v=hi8QQ89kzho [Accessed 11 Dec. 2018].

[10] Tzanidou, A. (2018, November 20). To tech or not to tech? [Web blog post]. Available at:  https://digitalcivics.io/to-tech-or-not-to-tech/ [Accessed 12 Dec. 2018].

[11] A. Yassine, S. Singh and A. Alamri, “Mining Human Activity Patterns From Smart Home Big Data for Health Care Applications,” in IEEE Access, vol. 5, pp. 13131-13141, 2017.

doi: 10.1109/ACCESS.2017.2719921 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7959184&isnumber=7859429

[12] Lekha R. Nair, Sujala D. Shetty, Siddhanth D. Shetty, Applying spark based machine learning model on streaming big data for health status prediction, Computers & Electrical Engineering, Volume 65, 2018, Pages 393-399, ISSN 0045-7906, https://doi.org/10.1016/j.compeleceng.2017.03.009.

[13] Mohamed Elhoseny, Ahmed Abdelaziz, Ahmed S. Salama, A.M. Riad, Khan Muhammad, Arun Kumar Sangaiah, A hybrid model of Internet of Things and cloud computing to manage big data in health services applications, Future Generation Computer Systems, Volume 86,2018,Pages 1383-1394,ISSN 0167-739X, https://doi.org/10.1016/j.future.2018.03.005 .

[14] Gunasekaran Manogaran, R. Varatharajan, Daphne Lopez, Priyan Malarvizhi Kumar, Revathi Sundarasekar, Chandu Thota, A new architecture of Internet of Things and big data ecosystem for secured smart healthcare monitoring and alerting system, Future Generation Computer Systems, Volume 82, 2018, Pages 375-387, ISSN 0167-739X, https://doi.org/10.1016/j.future.2017.10.045.

[15] Huda, M., Maseleno, A., Atmotiyoso, P., Siregar, M., Ahmad, R., Jasmi, K. & Muhamad, N. (2018). Big Data Emerging Technology: Insights into Innovative Environment for Online Learning Resources. International Journal of Emerging Technologies in Learning (iJET). 13 (1), pp. 23-36. Kassel, Germany: International Association of Online Engineering.

[16] Sionti, Marietta & Schack, Thomas & Aloimonos, Yiannis. (2018). An Embodied Tutoring System for Literal vs. Metaphorical Concepts. Frontiers in Psychology. 9. 10.3389/fpsyg.2018.02254.

[17] Tabea Turrini, Daniel Dörler, Anett Richter, Florian Heigl, Aletta Bonn, The threefold potential of environmental citizen science – Generating knowledge, creating learning opportunities and enabling civic participation, Biological Conservation, Volume 225, 2018, Pages 176-186, ISSN 0006-3207, https://doi.org/10.1016/j.biocon.2018.03.024.

[18] John M. Carroll and Mary Beth Rosson. 2013. Wild at Home: The Neighborhood as a Living Laboratory for HCI. ACM Trans. Comput.-Hum. Interact.20, 3, Article 16 (July 2013), 28 pages. DOI: https://doi.org/10.1145/2491500.2491504.

[19] Vlachokyriakos, Vasillis & Comber, Rob & Ladha, Karim & Taylor, Nick & Dunphy, Paul & Mccorry, Patrick & Olivier, Patrick. 2014. PosterVote: Expanding the action repertoire for local political activism. Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS. 10.1145/2598510.2598523.

[20] Clara Crivellaro, Rob Comber, John Bowers, Peter C. Wright, and Patrick Olivier. 2014. A pool of dreams: facebook, politics and the emergence of a social movement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3573-3582. DOI: https://doi.org/10.1145/2556288.2557100

[21] Oliver L. Haimson, Jed R. Brubaker, Lynn Dombrowski, and Gillian R. Hayes. 2016. Digital Footprints and Changing Networks During Online Identity Transitions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). ACM, New York, NY, USA, 2895-2907. DOI: https://doi.org/10.1145/2858036.2858136

[22] Weiser, M. 1991. The Computer for the 21 st Century. Scientific American, 265(3), 94-105. Retrieved from http://www.jstor.org/stable/24938718

[23] Genevieve Bell and Paul Dourish. 2007. Yesterday’s tomorrows: notes on ubiquitous computing’s dominant vision. Personal Ubiquitous Comput. 11, 2 (January 2007), 133-143. DOI=http://dx.doi.org/10.1007/s00779-006-0071-x

[24] Kamberov R., Granell C., Santos V. 2018. Use Case Scenarios of Dynamically Integrated Devices for Improving Human Experience in Collective Computing. In: Rocha Á., Adeli H., Reis L., Costanzo S. (eds) Trends and Advances in Information Systems and Technologies. WorldCIST’18 2018. Advances in Intelligent Systems and Computing, vol 746. Springer, Cham, 581-592 https://doi.org/10.1007/978-3-319-77712-2_54

[25] N. J. Goodall, “Can you program ethics into a self-driving car?,” in IEEE Spectrum, vol. 53, no. 6, pp. 28-58, June 2016. doi: 10.1109/MSPEC.2016.7473149

URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7473149&isnumber=7473124

[26] Finch, Kelsey and Tene, Omer, Smart Cities: Privacy, Transparency, and Community (April 3, 2018). Cambridge Handbook of Consumer Privacy, Eds. Evan Selinger, Jules Polonetsky and Omer Tene . Available at SSRN: https://ssrn.com/abstract=3156014

[27] Maurtvedt, M. 2017. The Chinese Social Credit System. Surveillance and Social Manipulation: A Solution to “Moral Decay”? (Master Thesis). UNIVERSITY OF OSLO. http://urn.nb.no/URN:NBN:no-63457

[28] Jodi Forlizzi and Katja Battarbee. 2004. Understanding experience in interactive systems. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques (DIS ’04). ACM, New York, NY, USA, 261-268. DOI: https://doi.org/10.1145/1013115.1013152

[29] Holvast J. (2009) History of Privacy. In: Matyáš V., Fischer-Hübner S., Cvrček D., Švenda P. (eds) The Future of Identity in the Information Society. Privacy and Identity 2008. IFIP Advances in Information and Communication Technology, vol 298. Springer, Berlin, Heidelberg

Image Sources:

[Cover] https://www.flickr.com/photos/53959560@N00/8398795151/in/photostream/ (CC BY 2.0)

[Cover] https://www.flickr.com/photos/kirbyurner/3660521353/in/photostream/ (CC BY 2.0)

[1] https://www.techspot.com/news/73770-ibm-world-smallest-computer-size-grain-salt.html

[2] https://www.techspot.com/news/73770-ibm-world-smallest-computer-size-grain-salt.html


Author: Tzanidou Alexandra

Leave a Reply

Your email address will not be published. Required fields are marked *