AuReRo: Augmented Reality Robotics

News and Events

• Research Grant -- Call for two positions (deadline: Dec 16th, 2011)

Executive Summary

Field robotics is the use of sturdy robots in unstructured environments. One important example of such a scenario is in Search And Rescue (SAR) operations to seek out victims of catastrophic events in urban environments. While advances in this domain have the potential to save human lives, many challenging problems still hinder the deployment of SAR robots in real situations. This project tackles one such crucial issue: effective real time mapping. To address this problem, we adopt a multidisciplinary approach by drawing on both Robotics and Human Computer Interaction (HCI) techniques and methodologies.

To achieve effective human-robot interaction (HRI), we propose presenting a pair of stereo camera feeds from a robot to an operator through an Augmented Reality (AR) head-mounted display (HMD). This will provide an immersive experience to the controller and has the capacity to display rich contextual information. To further enhance this display, we will superimpose mapping data generated by SLAM-6D (Simultaneous Localization And Mapping) algorithms over the video input using Augmented Reality (AR) techniques. Three modes of display will be available: (1) a top view 2D map, to aid navigation and provide an overview of spaces, (2) a rendered 3D model to allow the operator to explore a virtual model of the environment in detail, and (3) a superimposed rendered 3D model, synchronized with the field of view of the operator to form an augmented camera view. To achieve this latter mode we will track the angular position of the HMD and use it to both control the orientation of the robot's cameras and the rendering of the AR scene.

We will use real firefighter training camps as an experimental setup, leveraging a prior relationship between IST/ISR and the Lisbon firefighter corporation. The cooperation with firefighter teams will also provide subjects for initial task analyses and later usability evaluations of the system in the field. Concerning the robotic platforms, we will use existing robotic platforms at IST/ISR (e.g., the RAPOSA robot, together with other commercial platforms), upgrading them with the necessary equipment to achieve the project goals: 3D ranging sensors, calibrated stereo camera pairs with a pan&tilt mounting, and increased computing power.

Partners

• Institute for Systems and Robotics (ISR), Instituto Superior Técnico (Lisbon, Portugal)
• Madeira Interactive Technologies Institute (MiTi) (Madeira, Portugal)

People

• Rodrigo Ventura (Principal Investigator, ISR)
• Ian Oakley (Researcher, MiTi)
• Filipe Jesus (MSc candidate, ISR)
• Pedro Vieira (MSc candidate, ISR)
• Felix Tener (IAESTE grantee, ISR)
• Keiya Okada (IAESTE grantee, ISR)

Funding

• FRARIM: Human-robot interaction with field robots using augmented reality and interactive mapping (PTDC/EIA-CCO/113257/2009)
• LARSyS Plurianual Budget (PEst-OE/EEI/LA0009/2011)

Related work

Projects

• RAPOSA robot
• Rescue project

Publications

• Immersive 3-D Teleoperation of a Search and Rescue Robot Using a Head-Mounted Display, Henrique Martins, Rodrigo Ventura, IEEE International Conference on Emerging Techonologies and Factory Automation (ETFA-09), Mallorca, Spain, 2009. — PDF
• Robust Autonomous Stair Climbing by a Tracked Robot Using Accelerometer Sensors, Jorge Ferraz, Rodrigo Ventura, International Conference on Climbing and Walking Robots (CLAWAR-09), Istanbul, Turkey, 2009. — PDF
• Autonomous docking of a tracked wheels robot to its tether cable using a vision-based algorithm, Fausto Ferreira, Rodrigo Ventura, Workshop on Robotics for Disaster Response, ICRA 2009 - IEEE International Conference on Robotics and Automation, Kobe, Japan, 2009. — PDF
• A Search and Rescue Robot with Tele-Operated Tether Docking System, C. Marques, J. Cristovão, P. Alvito, Pedro Lima, João Frazão, M. Isabel Ribeiro, Rodrigo Ventura, Industrial Robot, Emerald Group Publishing Limited, Vol. 34, No.4, pp. 332-338, 2007 , 2007. &mdash PDF
• RAPOSA: Semi-Autonomous Robot for Rescue Operations , C. Marques, J. Cristovão, Pedro Lima, João Frazão, M. Isabel Ribeiro, Rodrigo Ventura, Proc. of IROS2006 - IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, 2006. — PDF