Agile Justin has sensors in its two, four-fingered robot hands. It uses them to 'feel' objects. The robot recently demonstrated its skills at the automatica trade fair. Robotics researchers consider dexterity to be a major challenge. "Using new methods of adaptive artificial intelligence, we have succeeded for the first time in demonstrating abilities with the highly developed mechatronic DLR hand that come close to human dexterity.

In particular, rotating and manipulating objects without vision inputs and with an open hand was a challenge," says Berthold Bauml from the DLR Institute of Robotics and Mechatronics in Oberpfaffenhofen. Bauml's AI research has been advancing the development of the humanoid robot Justin for many years. Justin catches balls thrown to him, for example, or recognises materials by sensing them with its fingers. Now, adaptive AI has significantly expanded Justin's capabilities.

"Turning the object is good; dropping it is bad"
The researchers used what is referred to as deep reinforcement learning. This means that Justin learns what is correct or incorrect completely on its own, simply by being told what to do. In this application, for example, it was instructed that 'turning the object towards the target is good', while 'dropping the object is bad'. Machine learning of skills takes only two to three hours in a simulation; then Justin masters the task. It develops the movement strategy for the fingers in a similar way to a human, through trial and error. "The robot can then directly implement its solutions in the real environment. Even we as researchers are often surprised. The possibilities of our new adaptive AI methods are fascinating," says Berthold Bauml.

Thousands of examples for tens of thousands of things
The researchers also achieved another first in AI robotics. Justin is able to grasp arbitrary objects with its multi-fingered hands. For this, the researchers solved two major problems using AI. Firstly, using 50,000 examples, the robot learned what is referred to as a 'foundational model', which allows it to infer the shape of the back of an object from the visible front. Justin has thus acquired an understanding of three-dimensional shapes.

Secondly, Justin can then find an optimal position for its hand and adjust its twelve finger joints appropriately. Humans grasp objects intuitively. In order to convey this intuition to the robot, another AI system was fed with thousands of examples for tens of thousands of objects. "The computations took a few days. But now the robot can recall the correct grip within fractions of a second," explains Bauml. At automatica, Justin demonstrated its dexterity with objects handed to him by the visitors to the exhibition.

"The work on AI-based dexterous manipulation represents a global scientific breakthrough. Never before have hands been able to manipulate such a large variety of objects so quickly and so precisely. This success was only possible through the combined use of the latest AI methods with model-based methodological approaches and many years of robotics experience," says Alin Albu-Schaffer, Director of the DLR Institute of Robotics and Mechatronics.

"Until now, it was often assumed that you needed a very large amount of data to learn dexterous object handling with robotic hands. In contrast to data for large language models, experimental data in robotics, even if generated by simulation, is very costly. The DLR research team has found a way to require less data by using more robotics knowledge and a new learning architecture. Using their new approach, the team can even train these models on a single computer and does not require a large computing cluster."

Possibilities for industrial production and healthcare
The next big step in robotics could be in industrial manufacturing. Multi-fingered robotic hands could assemble very complex and small parts if they master flexible, standardised intelligent manipulation. DLR researchers are working on this in the 'SmartHand' project, for example.

Robotic assistants are also an important topic in healthcare. In the future, they could assist caregivers and give people with reduced abilities back a higher degree of independence in everyday life. The intelligent manipulation skills developed at DLR are also fundamental to this. These skills apply, for example, to the safe grasping cups, bowls, boxes or other diversely shaped household objects.

Related Links
DLR Institute of Robotics and Mechatronics
All about the robots on Earth and beyond!