Proceedings of the First International Symposium on Humanoid Robots, Waseda University, Tokyo, October 30-31, 1996, pp. 53-62


Humanoids: Future Robots for Home and Factory

K. Kawamura, D.M. Wilkes, T. Pack, M. Bishay, and J. Barile
Intelligent Robotics Laboratory
Vanderbilt University
Nashville, TN 37235, USA

Abstract

Humanoid robots have typically been thought of as service robots that work in a close relationship with one or more humans; however, we believe that humanoid robots can also play an important role in manufacturing, especially holonic manufacturing. In this paper, we discuss several key problems that must be solved before humanoid robots can assume useful roles in society, both at home and in the factory. We identify three basic classes of technologies that are needed to produce useful humanoids. The first consists of those key technologies that are needed for all humanoid robots, both for home robots as well as factory robots. The second class of technologies is more necessary for humanoid robots functioning in a home setting, while the third class is more appropriate for humanoids in the factory. In particular, we discuss in some detail two key technologies of the first class that we are developing in the Intelligent Robotics Laboratory at Vanderbilt University. The first is a simple and effective visual servoing method for achieving hand-eye coordination and the second is the Intelligent Machine Architecture (IMA), a software architecture for robot development. Additionally, we present two examples of how humanoid robots can be used in the home (as an assistant for the physically disabled) and in the factory (as an assembly holon).

1 Introduction

Humanoid robots have a unique opportunity to become useful assistants for society following the tradition of automobiles and personal computers. These technological devices must, however, address several key issues before they roll out of our research labs into useful roles in society. Humanoid robots must deal with human and machine interactions on many levels from physical touch to gestures and spoken languages. They must make their users and other humans feel comfortable with them. Humanoid robots should fit into normal living or factory environments with limited modifications so that they can work alongside their users and coworkers in homes, hospitals and factory floors. In return, society should make appropriate adjustments to build a humanoid robot infrastructure that helps simplify the robot structure and reduce its cost.

Why humanoid robots? What are the goals for humanoid robots? What kinds of breakthrough technologies do we need? For example, we have been trying to give robots "human-like intelligence" for a long time, but this goal remains elusive. From our experience in the Intelligent Robotics Lab at Vanderbilt University, there is no fundamental breakthrough that will suddenly yield working humanoid robots. What is needed is a dedicated effort on the part of researchers to address the problem of developing integrated solutions. Systems integration is given second billing in robotics literature and this must be changed. Conferences such as HURO '96 will go a long way to help produce a paradigm shift in robot development.

Humanoid robots have been almost exclusively tied to service robots; however, we believe that they will provide an important contribution to manufacturing as well. From the days of the Industrial Revolution, manufacturing has evolved from a labor and individual skill-oriented enterprise to technology-oriented mass production processes. Current trends are moving toward flexible manufacturing with the goal of developing customized products in small batches [1]. To respond to these requirements, production facilities will need to become reconfigurable and based on increasingly intelligent autonomous modules that dynamically interact. This is a goal for an assembly holon we are developing at Vanderbilt.

In this paper, we discuss several key issues that must be addressed before humanoid robots can assume useful roles in society, both at home and in the factory. We identify three basic classes of technologies that are needed to produce useful humanoids. The first consists of those key technologies that are needed for all humanoid robots, both for home robots as well as factory robots. The second class of technologies is more necessary for humanoid robots functioning in a home setting, while the third class is more appropriate for humanoids in the factory. The paper is organized as follows: Section 2 considers the goals and key technologies that are needed to produce working humanoid robots. Section 3 discusses two examples of how humanoid robots can be used in the home (as an assistant for the physically disabled) and in the factory (as an assembly holon). Section 4 contains the conclusions.

2 Goals and Key Technologies

A pictorial representation of the key technologies needed to produce working humanoid robots is given in Figure 1. The technologies shown in the upper part of the chart are needed primarily for robots working in a home environment while those in the lower part are most important for those working in a factory environment. Of course, this chart is not the final assignment of these technologies, and no doubt over time the relative positioning of these technologies in the chart will change. In this section, we discuss some of these technologies with particular emphasis on human-robot interaction, hand-eye coordination, and system integration. Human-robot interaction through an appropriately designed and natural user interface is a key consideration in robot design and has the ability to significantly amplify the usefulness of the robot. Hand-eye coordination, or visual servoing, is a fundamentally important capability that is useful for a broad spectrum of tasks. Finally, it is difficult to overstate the importance of developing an underlying software architecture that is flexible enough to support a wide range of capabilities that may need to be incorporated into the robot's design.


Figure 1: Technologies needed for home and factory humanoids

2.1 Human and Robot Interaction

Interaction between humans and robots must be studied in three forms: the content, the media, and the hardware [2]. Different humanoid robots will need different interactive features, as not all will be used for the same task. Determining which of these features to give a humanoid robot depends on several issues:

Successful communication between human and humanoid robots will only occur when the interactive capabilities of the robots are well suited to all three considerations. Obviously, some interactive skills work well in certain situations but fail in others. For example, speech recognition is useful in quiet environments with users who have no speech impediments, but are of little use on noisy factory floors or with users who cannot speak.

Humanoid robots for home or domestic use will work closely with users in casual and un-structured situations. Therefore, one of the most important considerations for the success of humanoid robots is the psychological aspect of human/robot interactions [3]. If a user can relate to a robot in natural media such as speech, the psychological impact of working with a robot is softened.


Figure 2: ISAC with User Interaction Devices

Natural interaction can be facilitated by sending and receiving information in human-like ways. Both human to humanoid (input) and humanoid to human (output) mechanisms must be developed. Current research at the Intelligent Robotics Lab focuses on user input modalities such as color face and hand tracking, gesture recognition, eye tracking, and speech recognition for interaction with a humanoid for aiding disabled persons (see Figure 2). DEC Research has combined facial animation speech synthesis into "Smart Kiosks" to use the expressive nature of human faces for talking with and eliciting responses from users [4,5]. More task oriented forms of interaction such as learning human skills by demonstration have been addressed by several researchers [6,7]. Other methods, such as emotion sensing, may allow humanoids to act based on unintentionally communicated information and thus relieve user frustration [8].

In both industrial and home settings, a team of humanoids must work together with other intelligent machines to accomplish common goals. Architectures and protocols for doing this in an efficient and unambiguous fashion are being developed [9]. Interaction between these entities will involve issues such as:

In industrial and institutional settings humanoids must work closely with humanoid robots. In these instances, more formal communications are needed. Artificial or formal languages can be introduced to reduce ambiguity and users will have more training. This language will have features and constructs for representing manufacturing agents, subtasks, constraints, and resources [9].

2.2 Hand-Eye Coordination

In order to have a human-like posture, humanoid heads take the form of active stereo camera heads. Other sensors such as auditory sensors could be mounted on these heads. Coordination of the head and hand motions based on sensory information is a current research issue at the Intelligent Robotics Lab. In that context, we are working on these issues: first, fixation of auditory and visual stimuli and second, grasping or reaching for the source of the stimulus. Hand-eye coordination (i.e., visual servoing) is indispensable for humanoids because it provides a means of continually monitoring and guiding the humanoid hand while performing a task. In addition to aesthetic considerations, an independent active camera head has the following advantages: observation of the gripper of the robot arm and a large field of view.

We present below a novel approach called Fixation Point Voxels (FPV) for hand-eye coordination [10]. We show that FPV greatly simplifies visual servoing. In addition, we explain how it relaxes the accuracy required of the tracking system, since FPV does not need accurate tracking of the gripper. FPV is based on target fixation by the left and right cameras. According to Ballard and Brown, Fixation Creates an Object-centered reference frame in [11]: "The fixation frame allows for closed loop behavioral strategies that do not require very precise three-dimensional information. For example, in grasping an object, we can first fixate the object and then direct the hand to the center of the retinal coordinate system. We refer to this behavior as the 'fixation point' strategy (Figure 3)". Ballard asserts that biological and psychological data strongly suggest the existence of an object-centered frame (B in Figure 3). This frame is selected by the observer and is centered at the fixation point. FPV is a realization of these ideas.


Figure 3: Fixation point strategy (from Ballard and Brown [10]).

Target fixation is a human-like behavior which has been pursued by several researchers. Besides being essential for the FPV approach, target fixation has the following advantages in the context of visual servoing. Fixating a moving target by continuous tracking [12,13] stabilizes the image of the moving target. Additionally, the optical flow of a tracked target is reduced to very small values compared to the rest of the scene. Such a discrepancy in the optical flow could be a means of segmenting the image. Finally, space-variant cameras and algorithms [14,15,16,17] require target fixation in the center of the camera where the resolution is maximum.

In the Fixation Point Voxels (FPV) approach, the camera head fixates the target in the center of the left and right cameras. Therefore, the left and right camera optical axes intersect at the target as shown in Figure 4.


Figure 4: The eight voxels used in the Fixation Point Voxels approach.

FPV is based on the fact that the fixation frame divides the space into 8 voxels, namely (see Figure 3): A1A2, A1B2, C1C2, C1D2, B1A2, B1B2, D1C2, D1D2. The gripper is in one of these voxels at any point in time before it reaches the target. Notice that the detection of which voxel the gripper is in is done by detecting which image quadrant the gripper is in for both the left and right images. This implies that tracking the gripper reduces to simply detecting the image quadrant in which it is projected. As an example, if the projection of the gripper to the left image is in quadrant A1 and in the right image it is in quadrant A2, then it is in voxel A1A2, and so on. To guide the gripper to the target (at the origin of the fixation frame), FPV uses very simple rules which merely depend on which voxel the gripper is in, as follows:

IF A1A2 THEN (Right, Back, Down)

which means if the gripper is in voxel A1A2 then it should be moved in the direction (Right, Back, Down). Other rules are:

IF A1B2 THEN (Right, Forward, Down)
IF C1C2 THEN (Right, Back, Up)
IF C1D2 THEN (Right, Forward, Up)
IF B1A2 THEN (Left, Back, Down)
IF B1B2 THEN (Left, Forward, Down)
IF D1C2 THEN (Left, Back, Up)
IF D1D2 THEN (Left, Forward, Up)

Such a rule-based behavior greatly simplifies the implementation of hand-eye coordination in Humanoids. For more details on the implementation and the results (see [10]).

2.3 The Intelligent Machine Architecture (IMA)

The Intelligent Robotics Laboratory is developing the Intelligent Machine Architecture (IMA), a new approach for robot system integration. This approach is a synthesis of the many successful developments in behavioral robotics, software engineering and developments in control theory. The name is intended to reflect our desire to build intelligent-acting machines. The IMA can be characterized by the following properties:

Like most behavioral approaches [18,19,20,21], the system is built from behavior producing modules. Their outputs are combined using command arbitration and goal arbitration mechanisms. By selecting a behavioral approach we hope to reap the benefits of incremental development and implementation as well as the well known robustness of system operation that this approach offers [22]. Furthermore, in our experience it is easier to design a behavioral system because the description of the system more closely matches the problem at hand. One feature of IMA that departs from previous behavioral architectures is that behavior can be combined at many levels in the system, not just at the actuators, which reduces redundancy in the behavioral modules. For example, motion schema modules provide an arbitration mechanism and interface for several behaviors that move the arm, instead of each behavior directly moving the arm. This encapsulation makes the system more easily implemented and maintained from the development standpoint. Unlike many behavioral approaches, we use the behavioral approach as an engineering tool to build robust intelligent-acting robotic systems and do not claim that it is a model of human-level intelligence.

One instance of the IMA is shown below as an example of the behavioral control system that we are developing for our humanoid robot, ISAC [23], in the Intelligent Robotics Laboratory. This system combines the use of motor schema for manipulators, subsumption for tracking initialization and will use spreading activation for behavior sequencing in higher levels of the control system. By building our behavioral system within an object-oriented framework we can subclass generic behavior mechanisms and avoid redundant implementation of common system elements.


Figure 5:Basic Behavior System for ISAC.

The basic behavior system shown in Figure 5 for ISAC is intended to give our humanoid several basic capabilities that will help make it a useful robot for home and factory applications. Motion schema provide a flexible interface to the robots arms that supports the integration of constraints, a-priori knowledge and dynamic commands from the visual servoing system. Collision, force and handoff coordination behaviors contribute to intelligent motion of the arm by pushing the arms away from collisions, limiting forces in the arms and bringing the arms together to aid handoffs between the arms. Range schema push the manipulators away from singularities and can be implemented without inversion of the Jacobian, etc. of the arm by using this approach.

The sound fixation, and skin-tone fixation are designed to point the camera head at nearby people and initialize the tracking behaviors to track their hands and faces. The workspace-following behavior tends to keep the hands of the humanoid's arms in the visual workspace of the robot so that visual servoing can be easily applied. These with visual servoing provide a natural "follow the user" behavior, so that the robot will tend to mimic and follow human hand motions or head motions. The motion history of the sensory head can be used to detect gross gestures from a user. These behaviors provide the basic reactive substrate to which we will add behaviors specialized for tasks in the home or in the factory environment.

3 Application to Home and Factory

To illustrate our research approach and our vision of future humanoid robots in the home and the factory, we present two examples below.

3.1 Assistant for the Disabled

We are implementing an intelligent robotic aid system for physically handicapped people. The ultimate goal of our development is to build a small group of practical robots, each with specific structures and limited functionality. They will cooperate with the user and each other to provide a powerful and robust system. Our service robot system is based on collaborating semi-autonomous robots that interact closely with the user and each other. We use the "user in the loop" paradigm. If the robot has sufficient autonomy for its tasks, there is no problem. Should the robot become stuck or trapped, it requires help from the user. When the intelligence of the user is not integrated into the system, a service robot becomes less useful. Thus, the system design allows the user to command, guide, and override all of the robot's actions. However, the user should not teleoperate the robot., because his is too tedious, exhausting and frustrating. We need a balance of local autonomy and user intervention. We call this Human Directed Local Autonomy (HuDL).

Our testbed system consists of a humanoid robot and a mobile robot (see Figure 6). The humanoid is stationary and has dual SoftArm manipulators with a pan-tilt trinocular camera head and a central monitor for a graphical user interface (in the "belly" of the humanoid). The mobile robot is a modified version of a Yaskawa Helpmate mobile robot. It is fitted with a pan-tilt camera head and a manipulator. These two robots work together to serve the needs of the handicapped patient. For example, the mobile robot may be sent to fetch some food item and bring it to the humanoid in order to feed it to the user. Interaction is accomplished through such natural interfaces as speech recognition, sound localization, gestural interface, joystick, etc.


Figure 6: The humanoid robot test-bed in the Intelligent Robotics Lab.

3.2 Manufacturing - Humanoid as an Assembly Holon

In April 1995, a global program in Intelligent Manufacturing Systems (IMS) was inaugurated. It is designed to advance a technological and organizational manufacturing agenda to meet the challenge of a global manufacturing environment. In May 1996, we joined the Holonic Manufacturing System (HMS) project within IMS. A holonic manufacturing system is a manufacturing system having autonomous but cooperative elements called holons [24]. A holon can be comprised of other holons while, at the same time, being part of another holon. This gives rise to a holarchy where all holons autonomously manage their component holons and simultaneously allow themselves to be managed within the holarchy of which they are a part [9].

Our goals within the HMS project are to develop a holonic architecture for batch manufacturing tasks and to develop a prototype assembly holon. IMA is expected to be key to a holonic architecture. The dual-arm ISAC system will be easily adapted to become an assembly holon in the emerging agile enterprise [25]. The agile enterprise is capable of semi-custom "manufacturing to order" in an open, distributed environment. A vision of such agile enterprise is shown in Figure 7 below.


Figure 7: Intelligent Manufacturing Systems (from Dept. of Commerce, Coalition for Intelligent Manufacturing Systems).

4 Assistant for the Disabled

In this paper, we discussed several key issues that must be addressed before humanoid robots can assume useful roles in society, both at home and in the factory. Three basic classes of technologies that are needed to produce useful humanoids were identified. At the Intelligent Robotics Laboratory we are working on these key capabilities for our humanoid robot. We described methods for visual servoing, human-humanoid interaction and system integration. Two examples of how humanoid robots can be used in the home (as an assistant for the physically disabled) and in the factory (as an assembly holon) were shown. In conclusion, instead of searching for the single breakthrough that will result in humanoid robots we are developing the capability to integrate the many complex subsystems that will contribute to the future implementation of humanoid robots.

References

[1] G. Biswas, K. Kawamura, A.Saad, and M. Curtin, "Intelligent and Environmentally Conscious Manufacturing Systems-State of the Art," International Journal of Environmentally Conscious Design & Manufacturing, Vol.4, No.2, 1995, pp. 1-10.
[2] G. Biswas, T. Arai, M. Iskarous, Intelligence group report, in: Robotics and Autonomous Systems, Volume 18, Numbers 1-2, July 1996, pp. 225-234.
[3] P. Dario, E. Guglielmelli, V. Genovese, M. Toro, Robot assistants: Applications and Evolution, in: Robotics and Autonomous Systems, Volume 18, Numbers 1-2, July 1996, pp. 225-234
[4] K. Waters, T.M. Levergood, DECface: An Automatic Lip Synchronization Algorithm for Synthetic Faces, Digital Equipment Corporation Technical Report CRL 93/4, Sept. 1993
[5] Digital Research, Cambridge Research Labs Vision-based Interaction Group, WWW page, http://www.research.digital.com/CRL/projects/vision/plan.html
[6] M. Kaiser, R. Dillmann, Building Elementary Robot Skills from Human Demonstration, in: Proceedings of the 1996 IEEE International Conference on Robotics and Automation, April 1996, pp. 2700-2705
[7] M.C. Nechyba, Y. Xu, On the Fidelity of Human Skill Models, in: Proceedings of the 1996 IEEE International Conference on Robotics and Automation, April 1996, pp. 2688-2693.
[8] T. Sato, Y. Nishida, H. Mizoguchi, Robotic room: Symbiosis with human through behavior media, in: Robotics and Autonomous Systems, Volume 18, Numbers 1-2, July 1996, pp. 185-194
[9] G. Biswas, S. Bagchi, A. Saad, Holonic Planning and Scheduling for Assembly Tasks, Center for Intelligent Systems Technical Report CIS-95-01, 1995
[10] M. Bishay and K. Kawamura, "Fixation Point Voxels (FPV), A novel approach for head-eye-hand coordination," Submitted to IEEE International Conference on Robotics and Automation, 1997.
[11] D.H. Ballard and C.M. Brown, "Principles of animate vision," CVGIP, Image Processing, Vol. 56, No. 1, July 1992, pp. 3-21.
[12] D. Coombs and C. Brown, "Real-time binocular smooth pursuit," International Journal of Computer Vision, Vol. 11, pp. 147-164, Oct. 1993.
[13] S. Assad, M. Bishay, M. Wilkes, and K. Kawamura, "A Real-time intelligent vision sensor for robotics applications," IEEE International Conference on Robotics and Automation, April, 1996, pp. 1656-1661.
[14] M. Tistarelli and G. Sandini, "On the advantage of polar and log-polar mapping for direct estimation of time-to-impact from optical flow," IEEE Trans. PAMI, Vol. 15, no. 5, 1987.
[15] B. Bederson, R.S. Wallace, and E. Schwartz, "A miniaturized active vision system," 11th International Conference on Pattern Recognition, 1992.
[16] C.R. Weiman, "Log-polar binocular vision system," Final Report to NASA, Transition Research Corporation, Shelter Rock Lane, Danbury, CT, 1994.
[17] R.A. Peters II and M. Bishay, "Centering peripheral features in an indoor environment using a binocular log-polar 4 DOF camera head," International workshop on Human-Robot Symbiosis, Tsukuba, Japan, May 1995.
[18] R. C. Arkin, "Integrating Behavioral, Perceptual and World Knowledge in Reactive Navigation," Robotics and Autonomous Systems, Vol. 6, 1990, pp. 105-122.
[19] R. A. Brooks, "Integrated Systems Based on Behaviors," SIGART Bulletin, Vol. 2, No. 4 1992
[20] P. Maes, "The Agent Network Architecture," SIGART Bulletin, Vol. 2, No. 4, 1992
[21] L. Steels, "The Artificial Life Roots of Artificial Intelligence," Artificial Life, Vol. 1, No. 1, 1994
[22] R. A. Brooks, "A Robust Layered Control System for A Mobile Robot", IEEE Transactions on Robotics and Automation, 1989
[23] K. Kawamura, R. T. Pack, M. Bishay, and M. Iskarous, "Design philosophy for service robots," Robotics and Autonomous Systems, Vol. 18, no. 1-2, pp. 109-116, 1996.
[24] A. Koestler, A Ghost in the Machine, The Macmillan Company, 1971.
[25] S.L. Goldman, R.N. Nagel, and K. Preiss, Agile Competitors and Virtual Organizations, Van Nostrand Reinhold, 1995.

Center for Intelligent Systems | Intelligent Robotics Lab