NEC: AI TECHNOLOGY FOR ROBOTICS ENABLES PRECISE HANDLING OPERATIONS

NEC has developed AI technology for robotics that enables precise handling operations on unorganized and disorderly placed items. By predicting both the areas hidden by obstacles and the results of a robot's actions, this technology makes it possible for robots to perform tasks that were previously performed manually, thereby contributing to the improvement of productivity and work-styles.

Background
In recent years, due to labour shortages and other factors, the need for automation through the introduction of robots and large-scale equipment has been increasing in logistics warehouses and factories. However, it is difficult for existing robotics technologies to correctly recognize an environment in which objects and obstacles are disorderly placed, making it necessary to prepare an environment so that a robot can easily perform its tasks. For this reason, the introduction of robots has been limited to simple, routine tasks.

Features of this technology
NEC has developed AI technology for robotics that consists of two technologies based on "World Models" (*) - "Spatiotemporal Prediction," in which a robot precisely predicts the work environment and the results of its own actions from camera data, and "Robot Motion Generation," which automatically generates optimal and precise actions based on these predictions. According to NEC research, this is the world’s first technology of its kind to be applied to robot operations.



Application of World Models to Robotics

1. Autonomously executes precise actions in optimal sequences for items of various shapes and sizes
The handling of objects performed manually at a work site are executed by a combination of various actions. For example, in packing items, people can instantly execute a combination of precise actions such as "placing and then pushing items" without hitting other objects or obstacles. In robot control that uses conventional technologies, however, actions such as "push" and "pull" are more difficult to execute with high precision than actions such as "pick up" and "place." This is because slight differences in actions or shapes significantly influence how objects move in response to actions. In addition, as the number and types of actions to be considered increases, the combination and sequence of actions becomes more complex, which makes real-time planning a challenge.

This technology uses World Models to accurately predict the results of robot actions on objects of various shapes from video camera data, enabling robots to execute precise actions such as "push" and "pull." Moreover, robots can autonomously and instantly execute combinations of multiple actions such as "place and push" and "pull and pick up" by generating the appropriate action sequence at real-time speed depending on the work environment.

2. Operates while predicting hidden and invisible items
In a work environment where multiple items are closely arranged or disorderly piled up, people naturally predict the hidden areas and act accordingly, such as picking up items while avoiding interference with hidden objects. However, conventional recognition technology for robots has been difficult for practical use because it requires the preparation and learning of a large amount of teaching data showing the state of hidden objects in order to predict the hidden areas.

This new technology enables unsupervised learning that does not require labelling through the application of World Models and is able to efficiently learn prediction models of hidden object shapes. This enables robots to accurately predict a work environment from camera data and automatically generate optimal actions that do not collide with other objects or obstacles.



Technology Features

Future development
NEC will test this technology in logistics warehouses and other sites where much of the work is done manually by the end of 2024. By promoting the social implementation of this technology in various industries with a significant need for automation, NEC will contribute to improved productivity and workstyle reform.

(*) Technology that enables a robot to predict what will happen in the real world as a result of a certain action without trying it out in reality. This has been attracting attention in recent years as a key technology for autonomous control.