Haru is a social intelligent robot that provides a rich and multifaceted platform to investigate cognitive, emotional and empathetic engagement of humans with robots. As a learning companion capable of mimicking social and emotional behaviours, Haru can support children’s emotional and social skill development by connecting them to their peers all over the world, and facilitate children’s interactions to encourage self-expression through discussion and sharing in a series of engaging individual activities and group learning modules co-designed by the children themselves. Our study is to investigate how to build trustworthy children-centric robot that can familiarize children with the value of diversity, foster multicultural understanding and build children’s inter/intra personal skills and creativity with AI techniques.

It is difficult to program Haru to perform tasks in social settings because of the richness and uncertainty of environment. For Haru to be more useful and social, it must be able to learn to act autonomous quickly from ordinary people via natural interactions. In this study, we investigated and implemented methods for Haru to learn social behaviors from human provided rewards (evaluative feedback) via social reinforcement learning. This ability paves ways toward personalizing robot Haru that fit a child unique characteristics and enables robot Haru to implicitly understand children’s preference, so Haru can react accordingly in the long-term interaction process. Relevant publication: IEEE RO-MAN 2021

The tabletop robot Haru, used for affective telepresence research, enables a teleoperator to communicate affects from a distance. The robot’s expressiveness offers myriad ways of communicating affects through the execution of emotive routines. The teleoperator reacts to input modalities such as the user’s facial expression, gestures and speech-based intent as perceived by the robot’s perception system. However, due to the sheer number of routines to select from, the task of choosing the appropriate or the most preferred routine is becoming cumbersome. In this study, we propose a human-in- the-loop reinforcement learning mechanism in which an agent learns the teleoperator’s selection preference as a function of the input modalities and aids the routine selection process by narrowing it to n-best optimal choices. Relevant publication: IEEE ICRA 2021

The ability to understand the feelings of human users can make robots react more appropriately when collaborating with human beings. Implicit feedback such as facial expressions and vocalizations etc., are abundant, naturally occurring channels of information that human users provide during interactions. It would be immensely useful if robots can learn to understand human users, or perform social behaviors from such implicit feedback, which puts no additional burden on the human user. In this study, we adopt a human-centered reinforcement learning method to facilitate our social robot Haru to learn affective behaviors from human provided implicit facial feedback. Our results have been shared in two scientific papers and validated through peer-review process at the IROS 2022 and RO-MAN 2022 conferences.

Games play an important role in children’s education in the early learning stages. Many researchers have tried to develop social robots capable of playing games with children. However, in most of their work, the social robot’s level and strategy in playing the game are fixed, and cannot adapt to or learn from its human partners. In this study, we build upon the latest advances in the field of gaming AI and social robots by developing a social robot Haru to learn and imitate human strategy in a real-time two-player game via inverse reinforcement learning. Our results have been shared and validated through peer-review process at the ICSR 2022 conference.

Our goal is to develop a real-time adaptive emotional storytelling robot capable of building positive social relationships with users. In this study, we proposed a framework to facilitate social robot Haru to imitate the narrative of a human teacher for storytelling and adjust its style of storytelling according to the human teacher’s emotions, making use of human teachers as integral to the design of the system and providing a personalized storytelling experience. Our results have been shared and validated through peer-review process at the ICSR 2022 conference.