SHARE
Insights series
Accurate data for autonomous driving: Everything you need to know
What exactly does data accuracy mean in the context of training, testing, and validating an autonomous driving system? What does it take for one dataset to be more accurate than another? And why not having enough data accuracy could kill and bury your self-driving project.
These and more questions are going to be cleared up soon in our new insight series about data accuracy:
- The need for Pixel-accurate, synthetic data for autonomous driving perception, development & validation
- You can’t afford to use inaccurate data to train autonomous driving AI
- Seeking ground truth data generation… not going to happen using human annotators…
- How to generate accurate long-range detection data for AV – Facing the challenge
- A look into the future of data for training and validating autonomous vehicles
Real-world data is probably what came to your mind in the first place when you tried to answer what accurate data should be. And you are right, real-world data is perfect for our good ol’ seasoned human eyes. However, it doesn’t mean that real-world data is always the most convenient source of information to train artificial intelligence. The variables “who” is interpreting reality: perception systems; “how” they visualize it: sensor stack; and “what” they need to learn: the AI behind it, are absolutely key.
1. The need for Pixel-accurate synthetic data for autonomous driving perception, development & validation
Now that we are beginning to understand why data accuracy is key to a successful perception systems & new-generation sensors combination, it’s time to emphasize another important matter. Data accuracy has nothing to do with the concept of photorealism that we commonly attach to the images generated by off-the-shelf, real-time, computer graphics engines.
Planning on upgrading sensors?
Learn how to faithfully simulate your sensor and ISP and get full control to decide the best sensor for your perception system
Don’t get me wrong, these engines can develop beautiful, flashy images that can perfectly fit the requirements for other and less complex applications, but they don’t provide the precision to accomplish the data accuracy required for developing, for instance, human-safe and trustworthy, fully autonomous transport based on artificial perception.
Safety is one sine qua non condition AV/ADAS developers must commit to if they don’t want to end up working in a quicksand paradigm…
2. You can’t afford to use inaccurate data to train autonomous driving AI
This means that regarding these images as appropriate (from a photometric point of view) for training an autonomous transport perception system that requires the highest accuracy standards and minimal risk tolerance would be (if you’ll forgive the repetition) too risky…
In addition to the above, the images and data generated by real-time engines are also configured to be displayed on a low dynamic range device such as a computer screen or TV, and lastly, to be viewed by humans. It’s fair to say that, at the end of the day, these images are just “pretty” images, images that are qualitatively valid but don’t accurately simulate the behavior of a specific optical system, or a specific camera sensor…
3. Seeking ground truth data generation… not going to happen using human annotators…
To answer this question, let’s make a direct comparison between real-world data and synthetic data:
- Real-world data needs to be manually annotated and it is not an error-free task…
- Synthetic data introduces pixel-level, perfect annotations removing the possibility of error…
Leaving aside the fact that it is impossible to annotate real-world data at pixel-level, why apply a non-free-from-error human methodology to train systems which are more accurate than humans?
4. How to generate accurate long-range detection data for AV - Facing the challenge
As we have seen before, real-time graphic engines are able to provide labeled data, but since its generated data hasn’t been processed through an accurate, optical and sensor simulation, it still can’t guarantee pixel accuracy or physically correct metadata that the perception system may need for training and validation…
Training and validating AV deep learning models which face this technical challenge requires a major data generation and automatic labeling accuracy that only pixel-accurate synthetic data can offer today.
5. A look into the future of data for training and validating autonomous vehicles
If we look back, it’s amazing how many AV and ADAS systems have been developed with real-world data (and will probably continue to be used in combination with more accurate synthetic data), but real-world data limitations are more and more evident, to the point of questioning… Will future perception systems need them for training at all?
What happens when perception systems developers upgrade their cameras? Will they use their “legacy” real data to retrain the system? Will they capture new data? Or will they generate data simulating the new sensors? It’s a complex question and there is no simple answer, but accurate synthetic data is gaining more and more weight and is something you have to explore and keep an eye on.
About Anyverse™
With Anyverse™, you can accurately simulate any camera sensor and help you decide which one will perform better with your perception system. No more complex and expensive experiments with real devices, thanks to our state-of-the-art photometric pipeline.
Need to know more?
Visit our website, anyverse.ai anytime, or our Linkedin, Instagram, and Twitter profiles.
