ANYVERSE

CLIENT STORY

Improving LiDAR simulation accuracy with physically correct synthetic data

A strategic need

Generating the right sensor-specific data

To excel in an ever-changing and competitive perception for the autonomous machines marketplace, Cron AI identified the need to use simulation as a cornerstone of its long-term data strategy. Especially for the many applications that demand perception accuracy in extremely diverse environments and sensor configurations. Simulation allows them to overcome limitations in the datasets, especially for many hard-to-find corner cases of object types, poses, and distributions as well as to leverage perfectly labeled, complex object attributes that are impossible to annotate by hand. It also provides them with unique opportunities to generate highly project-specific data to scale for novel environments or sensor models before they have any access to them.

Cron AI & Anyverse partnership

IN NUMBERS

+ 7, 0 ,000

Detectable objects in the dataset

+ 0 ,000

Different point clouds in the dataset

+ 0 ,000

Different images in the dataset

Real world data vs Synthetic data

Real world data

Cron AI would’ve normally collected 1-2 sensors with real-world datasets

Synthetic data

Synthetic data allows them to simulate millions of different high-fidelity sensor variations

Real world data

Average object count with Cron AI real-world training datasets

Synthetic data

Average object count with the

synthetic dataset

Real world data

Usually, Cron AI could record 1, sometimes 2 synchronous sensors

Synthetic data

CRON AI simulated 9 synchronous sensors in the synthetic dataset

Real world data

Cron AI would’ve normally collected 1-2 sensors with real-world datasets

Average object count with Cron AI real-world training datasets

Usually, Cron AI could record 1, sometimes 2 synchronous sensors

Synthetic data

Synthetic data allows them to simulate millions of different high-fidelity sensor variations

Average object count with the

synthetic dataset

CRON AI simulated 9 synchronous sensors in the synthetic dataset

Would you like to read the full story?

Download PDF

Highest simulation fidelity required

Physically accurate

LiDAR simulation

A physically plausible simulation of LiDAR is extremely hard to come by. Most solutions, when they exist at all, are severely limited in the types of physical effects that can be simulated in the LiDAR’s light path. They are, therefore, severely limited in their usefulness to train deep learning models for real-world applications and adverse conditions.

Anyverse’s hyperspectral path, tracing technology in conjunction with their detailed approach to object material mapping and simulation, is very different. The simulation fidelity provided by Anyverse’s technology facilitates Cron AI’s proprietary post-processing pipeline to succeed in the creation of highly accurate simulations of light propagation.

Project objetives

The primary aim of Cron AI’s project is to use synthetic data to train the senseEDGE pipeline on as much diverse data as possible, and partnering it with Anyverse ensures that they have trained their neural networks on all possible corner cases that the real data may have missed.

The following key objectives were identified for this project:

Deliver pixel-accurate training data for the Cron AI sensorEDGE perception system

Leverage synthetic data based on a generic LiDAR simulation providing point cloud images with ground truth

Multiple layers of meta information for each measurement point to enable a deferred post-processing of sensor specific measurement artifacts

(Additionally) simulate an RGB camera, in the same position as the LiDAR, producing spherical images to match the point clouds

Randomly generate thousands of 3D scenarios with the placement of sensor riggings and positioning of objects of interest according to Cron AI project’s specs

Deliver pixel-accurate training data for the Cron AI sensorEDGE perception system

Leverage synthetic data based on a generic LiDAR simulation providing point cloud images with ground truth

Multiple layers of meta information for each measurement point to enable a deferred post-processing of sensor specific measurement artifacts

(Additionally) simulate an RGB camera, in the same position as the LiDAR, producing spherical images to match the point clouds

Randomly generate thousands of 3D scenarios with the placement of sensor riggings and positioning of objects of interest according to Cron AI project’s specs

Project challenges

Cron AI required a synthetic data partner which could have met the following challenges:

  • LiDAR ultra-high resolution
  • High number of objects of interest
  • Tailored distribution of objects and sensors that required several iterations with the customer
  • Dataset size (~2GB per sample)
  • Tailored distribution of objects’ poses

Approach & technology

This project needed a synthetic dataset focused on the information provided by a LiDAR sensor.

  • LiDAR and camera simulation
  • Procedural scene generation using Anyverse API (python)
  • Anyverse Hyperspectral Render
  • Automatic data generation in the cloud (scale)

In addition to the LiDAR point cloud, a camera image with a spherical projection (that allowed the points in the cloud to be correlated with the colors of the camera image) was necessary.

Would you like to read the full story?

Download PDF

The solution

Anyverse designed a comprehensive dataset empowered by its synthetic data generation platform to meet Cron AI requirements. Synthetic as well as urban environments, with city structures and all the necessary elements (with multiple variations to bring it to life), including characters with random poses throughout the entire scene.

The object placement in the scene was key to the project. Cron AI was very conscious of this point, the reason why Anyverse Studio™ (Anyverse’s platform module for developing scenarios) became critical, was due to its power to program, control, and customize the object placement in the scene.

Anyverse allowed them to control the positioning of each group of objects in each sample of the dataset, being able to control the quantity (for example from 2 to 5 groups of different objects), type (people, vegetation, benches or other furniture...), or size of these objects (buildings of different sizes), adding a minimum and a maximum number of objects, etc.

Results

Cron AI was able to successfully train their system on millions of variations of physically plausible simulated scenarios with a strong diversification of scene and realistic sensor properties that are prohibitively hard or even impossible to collect or diversify in any other way.

Simulated data was the perfect solution to mitigate dataset biases and is, therefore, a key ingredient to achieving a reliable performance of the system in new locations, or hardware and environmental conditions

Again, this turned out to be a very adaptive perception solution for Cron AI customers that does not break with data domain shifts due to new locations, hardware, and environment.

Connect with us!

Do these challenges sound familiar?

linkedin (1) Group 12 instagram

Let's talk about synthetic data!