Fully Automatic Picking of Unknown Products From Bulk Material

With the use of 3D cameras, robots are working on their own

Published: Thursday, December 23, 2021 - 13:03

In intralogistics, there has been a real hype about robotics for some years now, whether in trade journals or at fairs. Most of them are classic six-axis articulated robots that are looking for their way out of a production environment and into logistics. The goal: fully automated small-parts picking.

The main driver here is the labor shortage, and the big challenge isn’t a technical component like the robot or the gripper, but the design of an overall economic process. Because robots can handle only a portion of the items in each assortment, there are parallel streams of goods and thus possible risks with regard to the flow of goods, inventories, synchronization, and consolidation.

The “autopick” picking robot from psb intralogistics GmbH in Pirmasens, Germany, meets this challenge. At its core, the fully automated solution for picking individual items consists of a robot with gripper, the IT network for the entire system, and a powerful image-processing system that’s equipped with two Ensenso 3D cameras from IDS Imaging Development Systems GmbH.

Application

The vision system acts as the eye of the robot. It detects grippable surfaces on the objects to be processed in the source container and calculates grasping points and collision-free paths for the robot. This enables the robot to pick up unknown products directly from bulk material and place them in the target bin in the area with the lowest filling. Vacuum cups ensure gentle gripping of the respective objects.

A previous “teach-in” of the individual products isn’t necessary. Regardless of whether it’s a medicine bottle or a tea pack, the multifunctional gripper can be individually designed for a wide variety of articles with the most diverse packaging units. The system learns over time which of the different grasps works best for each item. The achievable picking performance for a process-safe system is dependent on the characteristics of the gripping objects and ranges between 300 and 500 parts per hour.

The robot picks up unknown products from bulk material and places them in the target container at 300 to 500 parts per hour, depending on the gripping object.

Two Ensenso N35 3D cameras provide the system with the necessary image data. All Ensenso 3D cameras work according to the “projected texture stereo vision” method. Each model uses two CMOS sensors and a projector that projects high-contrast structures onto the object to be captured, even in difficult lighting conditions. Ensenso cameras operate using stereo vision, which imitates human vision. The result is a 3D point cloud as the basis for the required spatial object information.

To integrate the cameras into autopick, psb intralogistics used the Ensenso SDK. In addition to wizards for easy setup and to support camera calibration of the 3D cameras, it includes the option for GPU-based image processing for even faster 3D data processing.

It also enables the output of a single 3D point cloud of all cameras used in multicamera operation, which is required in this case, as well as the live composition of the 3D point clouds from multiple viewing directions.

Two Ensenso 3D cameras provide the system with essential image data

The first camera is installed above the source box in order to consider the latter as a collision object in the path-planning of the robot arm. The unknown parts are presented here, and the point cloud for searching the appropriate handle point is generated. The latter is done with the help of the Mikado ARC (adaptive robot control) software from isys vision. It combines Ensenso’s 3D stereo-vision camera technology with an easily configurable, adaptive robot controller. The result is a complete 3D robot vision solution for bin picking and parts handling with an autonomously operating robot, as supplied by autopick. Instead of following predefined, taught, and firmly defined paths, it orients itself independently in the workspace and reacts to every situation.

The second camera is above the target box. It searches for free storage positions. In addition, the Z-height of the box content is analyzed and taken into account when determining the storage position. It’s crucial that the target box is filled evenly and that the products are placed carefully. The latter is particularly important when picking fragile parts. The image acquisition is asynchronous to the movement of the robot to optimize the cycle time.

Multifunctional vacuum cups ensure gentle gripping of a wide variety of items

“We chose the Ensenso N35 for its compact design and high point cloud quality,” explains the system manager at psb intralogistics. “For the considered field of view, the N series is ideal. The configuration of the camera can be easily and precisely adjusted to the respective product portfolio, or even adapted during the process.”

Handle information and picking orders are exchanged via the interface to the warehouse management system.

Outlook

Online trade continues to grow, and competition within the e-commerce sector is tough. Innovative intralogistics technologies can help companies remain competitive. This is especially true for logistics companies, which struggle to find suitable and talented employees. The economic benefit of picking robots is therefore substantial and will remain so for the long term. Underlying solutions using artificial intelligence will enable picking robots to master other challenges besides gripping, either independently or collaboratively with humans, reliably. Image processing using 3D cameras, among others, provides the decisive overview and the necessary safety.

About The Author

Sabine Terrasi

Sabine Terrasi manages corporate communications for German camera manufacturer IDS Imaging Development Systems.