Store Autopilots: Developing Retail Trade Using DSO-Based Navigator Drones

Introduction

Automation, a key trend in modern life, is pivotal in retail. Efficiently locating and delivering goods within stores and warehouses is an important part of the work of any store and warehouse. Time is crucial in these processes, and the solution lies in creating a 3D map of the premises for drone-based delivery of goods.

Research

To create a map from a movement video, we must measure the distances to objects in each frame. Approaches include tracking specific features like corners or bright spots, analyzing the entire image, or monitoring randomly chosen textured segments. However, the uniform and smooth warehouse walls complicate manual feature marking.

Initially, we explored feature detection, tracking fewer points to minimize the computational load. Yet, this focused excessively on minor details like small corners on shelves, whereas a broader premises map was required. Another method involved using the entire image or tracking random areas, but this risked overlooking small objects and protrusions, potentially leading to drone errors. To address these challenges, we turned to Direct Sparse Odometry (DSO), a more efficient and comprehensive mapping solution.

Stage set videos

Modern drones, equipped with diverse cameras and sensors, enhance distance measurement accuracy. Our focus was on testing basic monocular videos, captured using a GoPro, featuring clear movements, distortions, and rotations. To ensure accuracy, we emphasized the importance of camera calibration, either through direct calibration or post-processing alignment to eliminate distortion.

After aligning the video, the default parameters will do, and even narrowing or expanding the image will not significantly affect the operation of the algorithm we have chosen. Thus, we get from 3 videos 3 datasets with PNG files of size 1280×1024, without the fisheye effect.

DSO principles and approaches

The algorithm employs “optical flow” and “structure from motion” principles, enhanced with a direct photometric method. It segments the image, selecting pixels for tracking based on intensity gradients, which ensures an even distribution across the image and lowers power usage by not processing every pixel. This approach builds a relative map, not requiring exact distances, which suffices for our purposes. Notably, the algorithm adeptly handles sharp turns and rapid rotations, creating precise movement trajectories. Its versatility allows it to be easily ported and run on various OS and environments.

While effective, DSO does have limitations. Since it constructs a relative map, it could struggle with loop closures, where the drone revisits previously mapped areas, and outliers, potentially leading to inaccuracies. Additionally, DSO doesn’t save maps separately; each new environment requires adaptation and fine-tuning by engineers to ensure optimal performance.

Launch

Map building

The algorithm, adapted for macOS Monterey on an i5 processor, begins map creation with the first five frames, showing environmental points and gradually clarifying shelves and structures despite camera vibrations.

Exploring the map

Processing a 44-second video, the algorithm mapped basic structures and object outlines, including pathways between shelves.

General remarks

The basic DSO package includes many adjustable parameters, and visualization is not a mandatory part of the work. If you disable it, you can get a faster working system with real-time map building. The smoothness of the original video on the display is achieved by reducing the overall processing speed and is included only for the demonstration of the ratio of video and map building. Disabling the speed parameter limits the visual assessment of the algorithm’s work, but does not affect map building.

Summary

The DSO algorithm has great potential for automating warehouse logistics. The flexibility and versatility of the environment allow you to select the best hardware solution for indoor and outdoor navigation for each client independently. The wide range of parameters gives us the ability to monitor processes or fully trust the algorithm. Based on DSO’s capabilities, specialists can effectively construct detailed 3D maps of any space, ensuring precise navigation and enhanced logistics management. The creation and maintenance of an autopilot based on this algorithm is a progressive step toward improving customer interaction.

you might also like…