Research on Internal State-Based Systems

A new methodology for evaluating utility preferences using internal state information is attracting much attention within the robotics community. This methodology is based on on-going research in the fields of biology, psychology, and cognitive science and attempts to capture preference information through the use of artificial emotions, drives, and motivations.

Traditional state-based systems focus on external state information, such as the number and type of percepts, etc. when using the current state to influence decisions. External state-based systems scan the environment and then react or deliberate using the information gathered. Internal state-based systems monitor the external state, but these systems also include internal variables such as emotions, motivations, and feelings when making decisions. The internal variables are derived from dynamic internal processes and from associations and recollections pulled from long-term memory.

There are several approaches to designing these systems. Some approaches use biological homeostatic principles ([Cos-Aguilera, et al., 2005; Gadanho and Hallam, 1998; Canamero, 1997]), others use complex emotion models in an overall cognitive architecture ([Franklin, 2000; Sloman, 2001; Minsky, 1987]), while still others use homeostatic models and internal drives linked with emotions to improve human-robot interaction ([Breazeal, 2003; Gockley, et al., 2006]). However, research by [Davis and Lewis, 2004] suggest that robots do not benefit from classical biological models of emotion, but rather a more natural analogy to emotive concepts such as affective control states should be used.

Current research in the Center for Intelligent Systems – Cognitive Robotics Lab – is to integrate aspects of these different approaches in a manner that enables more efficient control of a humanoid robot. This research uses a low-level homeostatic system and a high-level affective system integrated with episodic and working memory to create preferences and motivations derived from both internal system demands and external commands.

The homeostatic system uses three variables based on the three innate psychological needs presented in [Ryan and Deci, 2000]: competence, autonomy, and relatedness. From these internal variables a high-level signal is computed that represents the system’s internal well-being. This is coupled with a second high-level variable, affect, that results from the external rewards (and punishments) the robot receives during task execution. These internal variables are recorded in episodic memory so that they can be used to bias future decision-making. In addition, as percepts and situations become associated with states of the internal system this process influences the contents of working memory, which ultimately guides the robots deliberative decision-making. Figure 1 shows the design of the internal dynamic state system (IDS) and Figure 2 describes how the IDS is integrated with the ISAC cognitive architecture.

Figure 1: Internal Dynamic State System

Figure 2: Integration of IDS with ISAC Architecture

The internal dynamic state system requires intricate interaction with episodic long-term memory. Through this interaction, previous affective states are used to influence the current affective state. The addition of internal state information to the episodes in episodic LTM also provides another dimension for ISAC to create situational similarity. Furthermore, by encoding and retaining information such as state of the internal system before and after a behavior is executed, ISAC may be able to learn affordances from the environment that implicitly enable situations to be reached. Finally, by supplying internal state information to the reactive and executive processes, the IDS allows ISAC to choose between externally commanded goals and internal goals.


[Breazeal, 2003] C. Breazeal, “Emotion and Sociable Humanoid Robots”, International Journal of Human-Computer Studies, Vol. 59, pp. 119-155, 2003
[Canamero, 1997] D. Canamero, “Modeling Motivations and Emotions as a Basis for Intelligent Behavior”, Proc. of the First International Symposium on Autonomous Agents, 1997
[Cos-Aguilera, et al., 2005] I. Cos-Aguilera, L. Cañamero, G.M. Hayes, and A. Gillies, “Ecological Integration of Affordances and Drives for Behaviour Selection”, Proc. of MNAS2005, Edinburgh, 2005
[Franklin, 2000] S. Franklin, “A Consciousness Based Architecture for a Functioning Mind”, Proceedings of the AISB2000 Symposium on How to Design a Functioning Mind, University of Birmingham, 2000
[Gadanho and Hallam, 1998] S.C. Gadanho and J. Hallam, “Emotion Triggered Learning for Autonomous Robots”, SAB’98 Workshop on Grounding Emotions in Adaptive Systems, 1998
[Gockley, et al 2006] R. Gockley, R. Simmons, & J. Forlizzi, “Modeling affect in socially interactive robots”, Proc. of 15th Annual IEEE International Workshop on Robot and Human Interaction, London, UK. 2006
[Minsky, 1987] M.L. Minsky, The Society of Mind, William Heinemann Ltd., 1987
[Sloman, 2001] A. Sloman, “Beyond Shallow Models of Emotion”, Cognitive Processing, Vol. 2, No. 1, pp. 177-188, 2001