On the pulse: cutting edge research on the power of social touch

Humans are by nature social beings tuned to communicate and interact. The sense of touch represents the most complex, direct and intimate, channel of communication and a powerful means of connection between individuals. The development and diffusion of internet-based technologies and virtual environments offer new opportunities of communication and overcoming physical distance. However, the tactile aspects of human-machine communication are overlooked, with a specific neglect of how interpersonal affective touch can promote multisensory and emotional co-presence.

In this blog we discuss the role of affective touch during social interactions with digital others or machines. We present scientific evidence and papers, and interviews with experts.

Can you trust a robot?

Beyond our day to day interactions with Alexa, Siri and Rumba, humanoid robots are being designed for richer forms of social interaction (or should we say relationship?) with humans. We have tried to make them move, to speak and to show expressions in as human-like a fashion as possible. Ultimately, this is in the hope of making these robots be perceived as trustworthy. But the question we want to ask is: If engineering and computer science create the perfect humanoid robot, will this be enough to inspire people's trust? By definition, trust relies on a willingness to depend on another party to achieve goals that we would not be able to achieve alone. Importantly, we trust someone not only because of their functional abilities, but we also consider personal characteristics, such as honesty and benevolence. We can say that trust implies a state of vulnerability, in which I risk putting myself in the capable, caring hands of someone else.

Assuming the robot's hands are capable, can they also be caring? Care and compassion are based on empathy, which is the ability to put oneself in the other person's shoes and share their affective states. Empathy is like a mirror: it implies the possibility for both partners to feel what the other feels. So perhaps to trust a robot, and share one's vulnerability, we do not need a perfect robot, but that spark of uncertainty and vulnerability that animates people. Hence the question: can we give robots vulnerability? Can they be capable and vulnerable at the same time?

One of the most intimate channels for connecting with the other, sharing affective states, vulnerability and mutual trust is social touch: a static pressure or gentle stroking on the other's arm, hugs, hand-shakes, tickle, caresses. Here at CeTI (https://ceti.one/de/), Junior Professor Merle Fairhurst and her team investigate the power of this kind of touch (given, received, reciprocal) in communicating various meanings and emotions, to comfort or seek comfort, to share joy. We wonder whether this experience is also possible in human-robot interactions.

“To become a socially intelligent actor a robot must be able to sense, classify and interpret human touch and respond to this in an appropriate manner” says Dr. Merel Jung, who collaborates on the project from Tilburg University. But even if this were technically possible, how would it be experienced by people? Would it induce the human brain and physiological systems to react as for human-to-human touch or in a unique way?

Want to find out more? Watch this space for our research in the area of touch and trust. 

https://scholar.google.it/citations?user=R_vUXEwAAAAJ&hl=it&oi=ao

Vulnerable robots? A debate between psychology and technology

One of the characteristics of interacting with people rather than inanimate objects is uncertainty. The communicative exchange is a cycle of expectations, prediction errors and updating of mutual knowledge. This uncertainty makes both partners vulnerable, and perhaps, precisely for this reason, human. Is this vulnerability necessary to consider a robot as a social partner? Technically speaking, how do we possibly animate a machine with uncertainty and vulnerability? It seems the opposite of what engineering is trying to do.

This piece collects the Q&A from an interview on this topic with Dr. Merel Jung, researcher at the department of Cognitive Science & Artificial Intelligence at Tilburg University, Netherlands. With a background in psychology and a passion for multidisciplinarity, Dr. Jung works on social artificial intelligence to enable robots to interact more naturally with humans, with a focus on the touch modality as a means of social communication.

Q. As an expert in the psychological processes underlying human-robot interaction, what are the fundamental characteristics of a robot to which we attribute social characteristics, with which we can interact not only to do things but also to communicate, to relate?

A: It seems like people don’t need much to anthropomorphize objects, robots or animals. When I was a visiting researcher at the University of British Columbia in Canada, I contributed to the behavioural development and testing of affective robots called CuddleBits. These robot creatures were flurry balls with one degree of freedom: they could only move up and down to simulate breathing behaviours and convey different emotions. People were spontaneously imagining these creatures to be their pet, even if they did not resemble any animal specifically. That happens also with sounds: if people perceived the creature as a cat, its motorised sounds were interpreted as purring signals. These studies seem to indicate that just simple movements could be enough for people to attribute social properties to a machine. In another study that I was involved in, healthcare providers were interviewed  about the use of a seal robot for people with dementia. Healthcare providers reported that the robot seal could bring up memories of former pets, even if it’s rather unfamiliar to hold a seal in one’s lap. The imagination of the human mind is quite rich and powerful.

Q: To what extent is this a novelty effect? The first time I interact with something new I may tend to rely on previous knowledge to build up some expectations and interpret what’s going on. But in the long run, we perhaps need more human-like features to perceive them as social agents?

A: This has to do with the concept of uncertainty, that makes interactions with other people and animals interesting, as you never know what they will be doing. If you have a robot with pre-programmed behaviours and interactive schemes, at some point you get tired of it. This limited form of interaction can be effective for young kids or people with cognitive impairments for whom having some nice tactile experiences might be very stimulating. As researchers are trying to program robots that can be interesting in the long term, some are moving towards generative algorithms that go beyond purely pre-programmed behaviours but rather start from a few hand-crafted schemes and then generate new ones using deep neural networks. This may be a way to include unpredictability and flexibility in robot behaviours without the need to do a massive programming work where every behaviour should be specified and implemented from scratch. Making these behaviours recognisable, plausible and sense-making would still be a challenge: you don’t want random behaviours but communication between a human and a robot that understand and adapt to one another. This research is still in a very early phase, where researchers are trying to compare hand-crafted and generated behaviours associated with certain emotions, for example, and see whether they are similar and people can actually recognise them.

Another compelling perspective is behavioural mimicry between humans and robots. When humans interact, they align, gradually use the same words, postures, facial expressions, and share back channel cues. If I speak and you nod, I get that you are probably listening and understand what I am saying. If the robot stands still while I am talking, I may wonder whether it’s still working or not. All these small non verbal cues are important and might be crucial to relate to robots as social partners. Having a history is also crucial: in a repeated interaction between a robot and a human, if the robot responds as if it were always the first encounter, this would break the relationship. It would be interesting to extend the abilities of robots to personalise the interaction. For example with face-scanning technologies, whereby a robot recognises who is in front of it and engages in behaviours that are specific to that person and that build on past experiences.

Q: Let’s imagine that we can give robots some of the vulnerability that makes humans human. If a robot couldn't do something and disclosed its limitations, would we feel frustrated (as for a malfunction) or compassionate?

A: Sometimes people expect flawless machines. On the other hand we know this is not the case. We know we still have to pay attention while using a navigator, cause there may be blind spots it doesn’t cover. There is a misconception that AIs are not biased, whereas they are based on data and the data is probably biased because they are collected by humans. Back to robots, it may depend on the expectations users have. If a robot looks like a pet, they may not expect it to hold a conversation. Ultimately, it is a question of affordances: if a robot has legs you will expect it to be able to walk. Unless it does so very smoothly and precisely, this may reduce its credibility. On the other hand, a small humanoid robot that walks awkwardly and looks vulnerable can be perceived as a toddler and elicit the users' empathy.

But for now we are applying human social norms to robots, discussing what we expect from and how we interact with others with certain characteristics. Perhaps at some point we will need to create new social norms to guide human-robot interactions more specifically. For example, we may need new ethical principles to determine which types of touch are appropriate when interacting with a robot. How would we greet a robot? With a hug, a handshake? Can the robot take the initiative and touch a person on the shoulder, or can it only respond to human initiatives?

Want to find out more? Here are a few interesting papers on the topic. 

- Suguitan, M., Bretan, M., & Hoffman, G. (2019, March). Affective robot movement generation using cyclegans. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 534-535). IEEE.

- Suguitan, M., Gomez, R., & Hoffman, G. (2020, March). MoveAE: modifying affective robot movements using classifying variational autoencoders. In 2020 15th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 481-489). IEEE. Link for video demonstration: https://dl.acm.org/doi/abs/10.1145/3371382.3378202

- Bucci, P., Cang, X. L., Valair, A., Marino, D., Tseng, L., Jung, M., ... & MacLean, K. E. (2017, May). Sketching cuddlebits: coupled prototyping of body and behaviour for an affective robot pet. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 3681-3692). Link for videos: https://dl.acm.org/doi/abs/10.1145/3025453.3025774

- Jung, M. M., Van der Leij, L., & Kelders, S. M. (2017). An exploration of the benefits of an animallike robot companion with more advanced touch interaction capabilities for dementia care. Frontiers in ICT, 4, 16. 

- Jung, M. M., Poel, M., Poppe, R., & Heylen, D. K. (2017). Automatic recognition of touch gestures in the corpus of social touch. Journal on multimodal user interfaces, 11, 81-96.

Adolescence: when everything changes

Physical contact is crucial for early physical, cognitive and emotional development and we now know that it is also important for lifelong psychological well being. From early in life, infants show a fundamental need for closeness to their parents and carers. Affective touch can improve not only social abilities but also learning and cognitive abilities more broadly. As we grow up this affinity for touch follows gender-specific, cultural, ideological and ethical social norms. These norms influence how interpersonal touch is used, when it is perceived as pleasant and how touch is used to facilitate our relationships with others.

In this developmental journey, adolescence is a sensitive period for a person's physical and neurophysiological development: everything changes and reorganises, and creating bonds with others can be difficult. The role of touch in one’s personal and social life is something kids may not be used to reflect on or talk about at that age, but a better understanding of this fundamental means of social connection and communication can help their psychophysical and social well-being.

We started a collaboration with Leipzig International School to open a window on how adolescents experience touch in everyday life, for them to get some extra tools to understand themselves, and to contribute to the collective knowledge of the role of touch in adolescence.

Using the HandsOn App, we asked kids to try different exercises to understand “When, and by whom, and where is it most pleasant to be touched?”. We also hope to discover to what extent their relationship to touch is related to the type of attachment style, which is the set of expectations, attitudes, thoughts and behaviours with which a person creates emotional bonds throughout life, based on the very first interactions they had with their caregivers during infancy. For this, students responded to a series of surveys about their relationships with peers and family members.

Follow us for updates on the results of this ongoing study!