Seeking ground truth data… not going to happen using human annotators

SHARE

I need ground truth data.

I need annotated datasets.

These are two of the requests that – frequently and many times together – we receive through our web forms (more often than you might imagine).

Autonomous driving systems, and other computer vision-based use cases, get inputs to start learning and “seeing” things as they are supposed to be. These systems need a vast amount of data, along with the right patterns to be able to identify situations and act accordingly in the real-life environment it will face.

This is where data annotation enters the scene.

Seeking ground truth data generation

Developing autonomous transport systems with real-world datasets means these need to be manually annotated so their deep learning algorithms are capable of recognizing and giving meaning to every object contained in the dataset.

Why is the method these annotations were generated by absolutely key to achieving (or not achieving) ground truth data?

Nowadays you can find several annotation methods, from the “old” (and certainly not error-free) human annotation companies to the more recent and partially automated data annotation tools. Both options are widely used, and both represent a risk (at different levels) for autonomous driving deep learning algorithms to understand and interpret the real-life situations they will face and what the correct action for each one of those is.

But… What if I told you there were a third and error-free option?

Perfect pixel-level annotations

The most sophisticated AI algorithms, such as the ones recently used in self-driving transportation, and especially those dealing with long-range detection, need to reach pixel-level accuracy. There is no margin for error at this point and human annotators (for obvious reasons) can’t (and shouldn’t) take on this task.

Leaving aside the fact that it is impossible to annotate real-world data at pixel-level, why apply a non-free-from-error methodology to train systems which are more accurate than humans?

One way to go is to use deep neural networks that can generate pixel-perfect annotations. The paradox is that these neural networks need to be trained with real-world data that has to be manually annotated leading to a chicken and egg situation. How accurate can the annotation neural networks be if they are not trained with accurate enough data?

Fortunately, there is another option. Today, there are already synthetic data solutions capable of delivering datasets with this level of accuracy. Datasets that are much more than images and generate ground truth data for advanced perception use cases. No matter if you are doing object detection, semantic or instance segmentation, image classification, or if you need 2D or 3D bounding boxes, or depth information… This and more channels are “packed” in your ready-to-use dataset.

It may be time to give synthetic data a try, but not just any synthetic data… pixel-accurate synthetic data capable of mimicking the behavior of your self-driving system in the real world.

About Anyverse™

Anyverse™ helps you continuously improve your deep learning perception models to reduce your system’s time to market applying new software 2.0 processes. Our synthetic data production platform allows us to provide high-fidelity accurate and balanced datasets. Along with a data-driven iterative process, we can help you reach the required model performance.

With Anyverse™, you can accurately simulate any camera sensor and help you decide which one will perform better with your perception system. No more complex and expensive experiments with real devices, thanks to our state-of-the-art photometric pipeline.