Machine learning at the robot cell

Important data at a glance

Subproject title:	Machine learning at the robot cell
Duration:	2019 – 2020
Collaborative project:	no
Funding:	Subcontract Fraunhofer IWU within the project Merhabe
Researcher / Contact person:	Dr. rer. nat. Michael Schwarzenberger
Cooperation:	Fraunhofer IWU
Logline:	Supporting the commissioning of a robot cell on machine tools with a switch localization of the machine control panel and adaptive path planning.

Objective

A work system consisting of a robot cell and machine tools is subject to frequent changes and uncertainties. Therefore, the teach-in should be designed to be adaptive by machine learning methods using analogy observations from existing data. The robot cell should adapt to current conditions (installation site, workpieces, dynamics, ...) independently. The teach-in is particularly time-consuming and should relieve the operator by proposing solutions for adaptive path planning and automatic control panel recognition and switch localization.

Solution

For adaptive path planning, a pipeline is developed for automatic determination of collision-free spaces based on the robot camera and subsequent path planning and simulation using ROS (Robot Operating System).

Prozessabfolge_Bahnplanung — Fig. 1: Process sequence for comparison of path planning algorithms, shown in green is the prototype of a machine tool being loaded and unloaded by the robot.

Machine panel detection is done using CNN (convolutional neural networks) and classical MachineVision methods for localizing switches on the panel.

Konzept_Bedienfelderkennung — Fig. 2: Concept machine panel recognition and switch localization Results

Results

Due to the flexible application constellation between machine tools, sampling-based algorithms allow a fast path planning, which enables a reliable and collision-free movement of the robot cell.

class_activation_map — Fig. 3: Class Activation Map (right) for a process image (left); areas marked in red indicate a higher weighting of the image region by the ML model.

The control panel recognition was successfully implemented using transfer learning, i.e., the use of a pre-trained CNN, and a least-squares SVM. Using this approach, new control panels can be learned with quite little training effort. However, care must be taken to ensure a balanced database, otherwise incorrect features will be learned (see Fig. 3).

Switch localization uses classical feature detection methods to recognize switch positions based on a pattern image. This approach convinces with a simple teach-in for the operator and a fast as well as robust switch localization during operation (Fig. 4).