What in the Clouds – Thermal Object Tracking

Introduction

How easy could it be to find something? When you look for clothes in a closet, all the objects are static, you just need to find the one you need. When you look for someone in an empty square, you look for moving objects. But if the square is crowded with people, walking about their business, constantly moving, finding someone who is moving in search of you is a task that has frustrated many.

Let’s turn our gaze to the sky – the wind blows, clouds are moving, and birds, drones, and insects are flying in random ways. To recognize the desired object among all the changes in the sky is a more complicated task. And that is the goal.

Data Analysis

Let’s start by cutting off all the unnecessary. The more information is received, the more power the brain or computer needs to process it, but the information must remain effective. Let’s switch to the thermal spectrum. Grayscale info, each pixel contains only 0-255 value. At the same time, both living beings and robots have their own parameters, and clouds still remain in the shot.

The result is close to the thermal or IR videos that are used in nature discovering, night vision etc.

The object we need to track:

In fact, we use thermal video to find and track an object the size of a few pixels on the entire image.

Technology

This task consists of three parts: stabilization, object detection, and object tracking.

Object detection

In each individual frame of the video, we need to select exactly those areas that attract attention. For this, we chose blob detection from the open library opencv-python.

Here we see the frame where lots of areas were chosen to track. It is important to select blob parameters that would correspond to our desired objects.

For example, filterByColor says that we will choose a blob by its intensity and blobColor = 255 mentions that the algorithm will search for an object brighter than its environment. This code also allows us to limit the size of the desired object, the minimum distance between objects, etc. In that way we remove unnecessary objects from our focus.

Object tracking

In order to accurately track one object frame by frame, it is necessary to catch it again in each frame. And here for conducting a more accurate trajectory, you also need to adjust – here we create parameters that would track the history of the object and the scatter in pixels, on which the object in the frame can make a jump. It is important to track the object even if it is lost, and to restore it when it is identified again.

As you can see, due to the camera shaking in an attempt to follow the object, it moves across the frame. It’s time to connect stabilization to ensure smooth transitions that will be taken into account by the tracking mechanism.

Stabilization

Many methods have been developed for the task of stabilization, but those such as SIFT, SURF, and ORB are not suitable for our task, since they require anchor corners – stable features by which they will align everything else. The best stabilization among clouds was shown by the DENSE algorithm, which tracks a grid of points, without choosing a priority for any of them. When selecting the size of the grid, we rely on the needs – high speed of processing or high accuracy of stabilization. Thus, we can get sufficient stabilization even with live video translation.

Review

This research main point is a tricky challenge: pinpointing tiny objects amidst shifting clouds in thermal video. By combining blob detection, object tracking, and DENSE-based cloud stabilization, it achieves impressive results, even in live-translated video. This opens doors for diverse applications, from wildlife monitoring to drone security, proving its potential to revolutionize object tracking in dynamic environments. While further tweaks might optimize performance, the core technology stands poised to transform our ability to “see through the clouds.”

you might also like…