Aero / Research project

Aero: Capturing Motion and Touch
for Dexterous Robot Learning

Chestnut Robotics

Aero UMI + Aero Hand
Co-designing human demonstration capture and robotic execution.

Video System overview Website source
Figure 1. Motion and tactile observations during card interaction. Aero Hand (left) and Aero UMI (right), with tactile signals visualized below. 13-second continuous excerpt; normal speed; no audio.

Abstract

Learning dexterous manipulation from human demonstrations requires capturing both hand motion and interaction with objects. Differences between the demonstration interface and the robot introduce a challenge: how should the recorded action relate to what the robot can execute? We present Aero, a platform that co-designs a wearable demonstration interface (Aero UMI) and a five-finger robotic hand (Aero Hand). The two devices are designed around corresponding joint motion and fingertip contact surfaces, with joint and camera calibration connecting their observations. Aero UMI records motion and tactile observations and supports teleoperation of Aero Hand. The goal is to collect demonstrations with the execution embodiment in mind, providing a basis for studying learning from motion and touch.

System Design

A demonstration interface and a robot hand designed as a pair.

Aero UMI wearable demonstration interface

(a) Aero UMIHuman demonstration interface

Aero Hand five-finger robotic hand on a robot arm

(b) Aero HandDexterous robotic hand

Figure 2. The paired embodiment. Aero UMI captures finger motion and tactile observations during human interaction. Aero Hand provides a five-finger execution platform with 15 active degrees of freedom and tactile sensing.

01 Motion correspondence

Joint motion is considered on both sides of the system, so demonstration capture is designed around the target hand’s action space.

02 Contact correspondence

Corresponding fingertip contact surfaces connect the design of the wearable interface to the robot’s interaction with objects.

03 Calibration

Joint and camera calibration connect the physical devices and their observations within the demonstration setup.

Manipulation Demonstrations

Qualitative examples of teleoperation and contact interaction.

Cup grasping

Figure 3. Teleoperation. Aero UMI guides Aero Hand through a cup grasp, with both devices visible in the same scene. 10-second continuous excerpt; normal speed; no audio.

Thin-object pickup

Figure 4. Fingertip contact. Picking up a utility blade from a flat surface, shown on the wearable and robot sides. Separate takes, shown side by side; normal speed; no audio.

These demonstrations illustrate system interaction; they are not evaluations of an autonomous learned policy.

Resources

Further information about the hardware and this project.

Additional video: Aero System introduction3 min 39 s
This introduction combines interviews, physical footage, and concept visualizations.