LOS ALTOS, Calif., June 21, 2021 — Today, Toyota Research Institute (TRI) unveiled new robotics capabilities aimed at solving complex tasks in home environments. Specifically, TRI roboticists were able to train robots to understand and operate in complicated situations that confuse most other robots, including recognizing and responding to transparent and reflective surfaces in a variety of circumstances.

As TRI demonstrates in a new video, this robust system allows robots to make generalizations in a range of scenarios, including in different homes. The video, released on National Selfie Day, takes a tongue in cheek approach to capturing these new capabilities on film, as the robot is seen recording itself as it proudly performs these new skills around the house. 

“Our goal is to build robotic capabilities that amplify, not replace, human abilities,” said Max Bajracharya, vice president of robotics at TRI. “Training robots to understand how to operate in home environments poses special challenges because of the diversity and complexity of our homes where small tasks can add up to big challenges.”

While a human can easily differentiate between an object and its reflection, transparent or reflective items commonly found in the home befuddle today’s robots. Since most robots are programmed to react to the objects and geometry in front of them without considering the context of the situation, they are easily fooled by a glass table, shiny toaster or transparent cup.

“To overcome this, TRI roboticists developed a novel training method to perceive the 3D geometry of the scene while also detecting objects and surfaces,” continued Bajracharya.  “This combination enables researchers to use large amounts of synthetic data to train the system.”  Using synthetic data also alleviates the need for time consuming, expensive, or impractical data collection and labeling.

While no system is perfect, today’s announcement adds to the body of knowledge helping robots to reliably navigate and operate in home environments. This technical achievement enables a robot to quickly learn from "programmable data"— synthetic data to recreate and learn from past failures and is a promising milestone for TRI and roboticists everywhere.