Scaling synthetic training and testing data for enhanced vehicle simulation.
Today, automotive simulation and testing play a very important role in the process of developing cars, buses, two-wheelers, trucks, HGVs and other such vehicles. As such, recent advances in smart electronics, safety control systems and artificial intelligence (AI) are continually driving the evolution of automotive simulation technologies in order to develop safer, more efficient and more reliable vehicles.
RFpro is an engineering-grade simulation environment for the automotive and motorsport industries. The software is used for the development and testing of autonomous vehicles, ADAS, vehicle dynamics and human factor studies, and has been chosen by some of the world’s largest car manufacturers, tier one suppliers and sensor developers. Speaking at the recent CENEX event in September, rFpro’s Technical Director Matt Daley shares how the software is helping to advance vehicle perception development using high-fidelity synthetic training data created using physically modelled sensors.
“We are about driving simulation, and we exist to do three things,” he says. “Firstly, we model the real world and then allow your vehicles, plug-ins and sensors to interact within that virtual world. Then, we help you to scale your training and testing data on a huge scale. But, what do we mean by the real world? Well, we need highly complex real-world locations, not just simple things that have been designed. This includes all the different types of natural and synthetic lights, the built environment, all the things that are happening inside the simulation, and potential scenarios you may wish to test out, such as automated parking systems.”
According to Daley, rFpro has the biggest digital model library in the world, consisting of proving grounds, public roads, different types of racetracks and testing circuits, as well as all different types of specific testing locations and challenging environments. So far, the company has built 15 private proving grounds for 11 different OEM and Tier One companies across the world, having scanned and digitally built their own in-house testing locations.
SYNTHETIC TRAINING DATA
“We help to connect all the different parts of a simulation together,” Daley explains. “In the past, it was humans driving in a simulation, connected to vehicle models. But now, we have sensors. Our job is to develop highly detailed and accurately engineered inputs into virtual sensors to create a complex environment within which we can interact, involving traffic, other vehicles and road users and so on. RFpro is the central simulation that enables all of those things to be connected together, and we do this for many different types of use cases.”
Many systems now are AI-based, requiring huge volumes of training data in order to work effectively. This does not only involve virtual simulation, but also the production of highly detailed ground truth, involving semantic segmentations and depth buffers that must be joined together to make a training dataset to train
the AI.
“So what are the key things we need when making synthetic training data?” asks Daley. “We need locations, we need virtual models of physical world places, we then need to fill them with lots of different types of traffic. And that’s not just your simple passenger cars, but also emergency vehicles, utility vehicles, things that shine and reflect. We need all of those complexities that drivers and vehicles will experience on the real roads in the simulation. The next stage is to fill the virtual model with different objects, like pedestrians, flashing lights, different roadsides and so on. Then, we have to make all of this into something useful, so we build scenarios. You can do that either manually by placing cones, for instance, or you can write Python scripts to automatically change the size and shape of locations, rotate objects or change their colours. Doing this, you can start to create the thousands upon thousands of different tests and variations of your synthetic training data required for your AI models.”
SUPERIOR SENSORS
Within this synthetic training environment, you not only need cameras but also lidar and radar models, too. All these sensors need to be physically modelled, including the lens, colour filters, arrays, CMOS chips, ISP and so on. By building these up, we can achieve highly reactive scenes capable of producing full HDR simulations, where you can input the multiple sampling frames that are going into each part of a camera simulation, and physically model the way that the sensor is opening over time. This allows the addition of motion blur, rolling shutter effects, LED flicker, exposures and other effects that can influence the sensor’s identification and perception capabilities.
“Once you have developed the sensors you need, you can start to develop the training data to sit alongside it,” Daley adds. “All of this works together: automatic 2D and 3D bounding boxes, semantic segmentation, all of the objects identified in your training data. All of this is why simulation is a more powerful tool than purely relying on physical testing alone. Synthetic training data has taken up massively in the past few years, and is now at that engineering level that can make a real difference to the perception testing of your systems.”
