MIT researchers use wristband to capture hand motion for robot training

CAMBRIDGE, Mass. (AP) — Humanoid robots struggling with tasks like grasping a cup have gained a new training tool: an ultrasound wristband that sees inside the arm. Researchers at the Massachusetts Institute of Technology developed the device to collect data on the movement of muscles, tendons, and ligaments beneath the skin during everyday hand motions, according to a paper released Tuesday.

The wristband records the subtle internal mechanics of the hand and wrist as a person performs actions that robots typically find difficult. The resulting data set could help machines learn the dexterous manipulation that remains a major hurdle for robotics, the researchers said.

“Imagine people doing housework,” said Xuanhe Zhao, an MIT professor of mechanical engineering. “We can use the data obtained by our system to train a robot to do exactly that housework with this dexterous hand motion.”

The sensor captures motion that is invisible from the outside. Unlike cameras that track hand position or gloves that measure surface movement, the ultrasound wristband peers beneath the skin to record the coordination of muscles and tendons that produce precise finger and wrist movements. The researchers said this deeper view could give robots a richer training signal than surface-level sensors alone.

The research addresses a persistent gap in robotics. Machines have become adept at tasks controlled by software — analyzing text, recognizing images, operating in structured environments — but physical manipulation of objects remains difficult. A robot that can assemble a car part may still fumble when picking up a cup of water.

Zhao is among a group of scientists working to equip AI with better understanding of the physical world. As much of the tech industry has focused on language models and virtual assistants that operate on computer-based tasks, the MIT team is targeting the sensory data needed to interact with objects and spaces.

The wristband is still in the research phase, and the team has not disclosed a timeline for commercial use. The data sets collected from volunteers performing household tasks will be used to train robotic systems in simulation and eventually in physical robots.

“We can use the data obtained by our system to train a robot,” Zhao said. The approach could also be adapted for other types of motion, such as walking or lifting, opening a path toward robots that learn physical tasks by watching humans.