ICRA2012 - Workshop on Haptic Teleoperation of Mobile Robots: Theory, Applications and Perspectives

welcome / schedule / proceedings / registration / organizers & contact


For several applications like surveillance, search and rescue in disaster regions, and exploration, the use of a single or a of group of mobile robots is getting more and more common. Nevertheless, when the tasks become extremely complex and high-level cognitive-based decisions are required online (as, e.g., during exploration of very cluttered, dynamic and unpredictable environments for search and rescue applications), complete autonomy is still far from being reached and human’s intervention/assistance is necessary. In this context, haptic teleoperation systems, where a human operator commands a remote robot through a local interface and receive an informative haptic feedback, allow to exploit human’s intelligence to solve tasks too complex for nowadays robots.

This full day workshop will focus on the haptic teleoperation of a single or a team of mobile robots. Its main goal is to present recent results in the field and to establish a discussion on the technological, mathematical and psychophysical aspects of this problem.

Invited speakers

  • Seugmoon Choi, Dept. of Computer Science and Engineering, Pohang University of Science and Technology (POSTECH), Republic of Korea
    • Title: Research Issues on Mobile Haptic Interface for Large Virtual Environments
    • Abstract: The mobile haptic interface is a novel solution for providing haptic sensations in a large virtual environment where a user can walk around. The feasibility of mobile haptic interface has been demonstrated by several research groups, but a number of challenging issues still remain in order to fully integrate a mobile haptic interface into advanced large visual displays. This paper presents a review on the current research status on mobile haptic interfaces and related engineering and evaluation problems. In addition, the design and performance of the mobile haptic interface that has been developed in the speaker’ laboratory are described briefly.
  • Antonio Franchi, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
    • Title: Modeling, Control, and Haptic Steering of Multi-robot Formations
    • Abstract: This talk will overview some theoretical and experimental results in the modeling, control and haptic steering of multi-robot formations, with a special regard to the multi-UAV case. It is commonly acknowledged that the major strengths of a multi-robot system are both the resilience to single point failures and the possibility of parallelizing the execution of a given task. These properties can be fully exploited in coverage-like tasks, e.g., exploration, pursuit-evasion (a.k.a. "clearing"), and periodical monitoring (a.k.a. "patrolling"). These tasks, in turn, contain several control subproblems. All these tasks often requires to keep a certain optimal arrangement which can be achieved by using an appropriate formation controller. Among the major challenges in this case there are both the decentralization of the controller and the use of cheap and lightweight sensors, like cameras. In addition to that, the presence of one or more human co-operators may result extremely useful for the multi-robot system, especially whenever the task becomes particularly challenging and requires complex, cognitive capabilities, e.g., in search and rescue operations. A relevant problem in this shared control case is how to balance the robot autonomy with the human assistance. It has also been proven that the presence of a bilateral (haptic) connection between the human and the machine increases the human situational awareness and improves the quality of the human-robot cooperation. On the other hand, the presence of a force feedback makes the control of the whole system more challenging, e.g., in case long-distances between the co-operator and the robotic group.
  • Dongjun Lee, School of Mechanical & Aerospace Engineering, Seoul National University, Republic of Korea
    • Title: On the semi-autonomous teleoperation of multiple nonholonomic mobile manipulators
    • Abstract: Semi-autonomous teleoperation, in which some simpler tasks are attained by local autonomous control and more complex behaviors are achieved through human teleoperation, is promising particularly when a single human user needs to simultaneously teleoperate multiple mobile robots with a large degree-of-freedom.  In this talk, I will present some recent developments in my lab on the semi-autonomous teleoperation of multiple nonholonomic kinematic mobile manipulators cooperatively holding and transporting a common deformable object.  Basic (and compatible) behavior-primitives of multiple mobile manipulators are extracted/designed by using nonholonomic passive decomposition (NPD).  Passive set-position modulation (PSPM) is then utilized to allow a remote human user to teleoperate the commonly-grasped object with useful haptic-feedback while addressing the master-slave kinematic dissimilarity; the remaining behavior-primitives are autonomously controlled so that each robot can individually avoid obstacles with the grasping left intact.  Some simulation results will also be presented along with a discussion on possible future research directions.
  • Domenico Prattichizzo, Dept. of Information Engineering University of Siena, Siena, Italy
    • Title: Mixed human-robot formation control with haptic feedback
    • Abstract: In this talk we explore a new formation control setup, consisting of a human leader and multiple follower robots. The mobile robots are equipped with RGB- D cameras, and they should maintain a desired distance and orientation to the leader. Vibrotactile haptic feedback provided by haptic bracelets, guides the human along trajectories that are feasible for the leader-follower formation. Extensive numerical simulations and real-world experiments conducted using the Pioneer robot and Microsoft’s Kinect camera, show the effectiveness of the proposed designs.
  • Paolo Robuffo Giordano, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
    • Title: Shared Control of Multi-Robot Systems: Passivity, Decentralization, and Connectivity Maintenance
    • Abstract: This talk will cover some recent theoretical and experimental results in the new topic of Bilateral Shared Control of Multiple Mobile Robots, with a special attention to the case of Unmanned Aerial Vehicles (UAVs). In this non-conventional teleoperation field, a human operator partially controls the behavior of a semi-autonomous group of mobile robots by means of one or more haptic interfaces, and receives back a force cue informative of the swarm tracking performance and of additional properties of the surrounding environment (e.g., presence of obstacles or loss of group connectivity). This kind of systems is designed in order to enhance the telepresence of the operator and the quality of the human robot interaction, especially when applied to practical scenarios, like search and rescue, surveillance, exploration and mapping. The talk will first present an overview of the nature and kind of problems addressed within this research such as, e.g., how to design a stable bilateral interconnection between the user and a group of flying robots bound to maintain connectivity of the underlying interaction graph, but otherwise free to shape their formation because of additional constraints or sub-tasks. We will present theoretical and experimental results obtained on a group of quadrotor UAVs, and then discuss future research directions.
  • Jee-Hwan Ryu, School of Mechanical Engineering, Korea University of Technology, Korea
    • Title: Bilateral Control Architecture and Master Interface for Mobile Robot Teleoperation
    • Abstract: In this talk, we are going to discuss about bilateral control architectures and an intuitive master interface for mobile robot teleoperation. Mobile robot teleoperation systems have two major differences when it is compared with manipulator type master/slave teleoperation systems. The first one is kinematic dissimilarity. So, rate mode teleoperation is generally used to overcome this. Second, there is no physical interaction and the feedback force displayed to the human operator is usually based on the distance information to the obstacle from the mobile robot. Therefore, general bilateral control architectures and master devices, which developed for tele-manipulation, have been considered not suitable for mobile robot teleoperation. In this talk, new bilateral control architecture for mobile robot teleoperation will be introduced including position speed switching teleoperation architecture and variable force feedback gain control scheme together with multi-robot and multi-state teleoperation architecture. New master interface for intuitive mobile robot teleoperation will be introduced as well.
  • Cristian Secchi, Dept. of Sciences and Methods of Engineering University of Modena and Reggio Emilia, Reggio Emilia, Italy
    • Title: Passivity based teleoperation of a group of UAVs with time-varying topology
    • Abstract: This talk will address the problem of bilateral teleoperation of a group of UAVs. In order to make the behavior of the fleet as flexible as possible and to allow split and join maneuvers while avoiding instability the concept of tank is exploited. It is also shown how tanks can be used for passively dealing with inter-agents and with master slave communication delays. Human hardware in the loop simulation and experiments with real UAVs will be shown to validate the bilateral teleoperation architecture.
  • Hyoung Il Son, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
    • Title: Psychophysical Evaluation of Visual and Haptic Feedback in Teleoperation of Multiple Mobile Robots
    • Abstract: For teleoperation of (multiple) mobile robot(s), a better situational awareness of remote environments is crucial for good performance. Visual and haptic feedback are the most common ways to perceive the environments accurately using a vision system and a haptic device, respectively. In this talk, I will present benefits and effects of visual and haptic feedback on performance (especially, from human perspective) in teleoperation of multiple mobile robots (e.g., unmanned aerial vehicles (UAVs)). In details, the following three issues will be described with results by several psychophysical studies in the field of bilateral teleoperation of multiple mobile robots: 1) the contribution of force feedback to human performance; 2) the design and evaluation of haptic cueing to increase human operator's performance; and 3) the effect of visual feedback on human's situational awareness in outdoor environment.
  • Stefano Stramigioli, Dept. of Electrical Engineering, University of Twente, Netherlands
    • Title: Coping with time delays in flying robots
    • Abstract: Passivity and energy-based reasoning have shown to be effective in many engineering applications. In the presentation "port-based" methods will be presented to tackle issue related to telemanipulation of flying vehicles in the presence of varying time delays. Furthermore, novel physically based methods will be discussed to tackle under actuation of flying vehicles.

List of topics

The topics of this workshop are, but are not limited to:

Bilateral Teleoperation and Haptic interfaces

Human-robot interaction

Shared Autonomy for Single-/Multi-Mobile Robots

Formal Methods for stable interaction/control

Applications and recent experimental results

User studies and psychophysical evaluations