By Gabriel Haas, Senior User Experience Project Manager
If you visited our booth at CES 2022 or attended one of our recent customer innovation days, you may have had the chance to see Cerence Co-Pilot in action. While it was only one of several demos, it attracted a lot of attention. Why? Because it consisted of a highly immersive driving simulator that featured a two-seater platform with real car seats, a huge, curved dashboard, and a body that held the steering wheel, pedals, and a navigation display. It was an immense effort to build, and a considerable logistical challenge to transport, but it was well worth the effort to bring this exciting experience to our customers and partners.
Many consumer electronic products such as TVs, speakers, or even regular voice assistants can be presented well in a trade show booth, as companies can replicate the environment in which the devices will be used. The devices are placed in their intended setting: the customer's living room. With Cerence Co-Pilot, which is designed to support drivers and passengers, this is not nearly as easy. We knew that to best showcase the capabilities of our proactive AI, we must create a realistic driving simulation that put the visitor in an actual driving situation. First, we designed and implemented the driving simulation, introduced in part one of this blog post. In addition to the actual driving simulation, many other components were integrated, including the interactive dashboard, a microphone array and speech recognizer, and an eye-tracking camera. While this created the perfect foundation to showcase all of Cerence Co-Pilot’s amazing features, it also allowed us to do three important things: showcase products in a realistic and tangible driving experience, rapidly prototype new products and components, and run high-fidelity user studies in a controlled environment.
Showcasing products in a realistic and tangible driving experience
Our proactive AI considers three types of data to provide the driver with intelligent suggestions while driving: first, information from vehicle sensors such as the current battery level, tire pressure, or infotainment usage; second, cloud services, such as the traffic situation or the latest weather report at the vehicle’s destination; and third, personal information that a driver shares such as calendar appointments.
While in our simulation, the intelligent decisions and actions Cerence Co-Pilot offers the driver can be experienced first-hand within a realistic and safe environment. We can even control many elements within the environment, such as traffic or weather, to provoke situations in which the co-pilot works its magic and performs an action to help the driver. This allows the full Cerence Co-Piot experience to be shown in a short time frame without having to resort to recorded video, but instead while the visitor himself is in full control of the vehicle.
Rapid prototyping of new products and components
Not only do interested visitors or customers benefit from this driving simulator, but we do as a team. To design and develop a successful product, we need to be able to rapidly test out new ideas as well as new features. The effort needed to bring a prototype of any new technology and software into a real vehicle, which at the same time must be able to be driven safely on the road, is immense. With the help of our simulator, we can integrate new technologies quickly and re-create realistic driving situations without compromising safety. This enables us to make the right decisions for further development at an early stage and thus improve our products and get them on the road faster.
Running high-fidelity user studies in a controlled environment
Finally, we can use the simulator for user studies. The feedback of real users is irreplaceable and extremely important to develop products that meet the wishes and needs of our customers. User studies in a highly immersive and realistic driving simulator can partially replace studies in real vehicles, but above all, they offer possibilities that go beyond what is possible in a real vehicle. Because the entire environment is under our control, dangerous situations can be created and tailored to suit the situation that we are wanting to test without placing the study participant at risk.
Let’s assume we want to test a function that is triggered after an accident. The simulation allows us to create such a situation by manipulating traffic or weather and simulating the consequences of an accident in a physically correct way. We could have a computer-controlled car crash into the participant's car at an intersection and then present the user with support from Cerence Co-Pilot without ever putting them in danger, something that is not possible in a real car. By letting the participant experience the accident, they can give a much more valuable assessment of the usefulness and helpfulness of this feature compared to when the situation is presented without real context.
One simulator - countless possibilities
All these possibilities are given to us by having a flexible driving simulator that provides us full control over the virtual environment and makes it easy to integrate new technology. While a simulation can never fully replace actual experiences in the physical world, it sometimes offers even more possibilities than the real world. For us, it provides many advantages and benefits to showcase, create, and improve our products.
We are already working on integrating the latest technologies and upcoming products into our driving simulator and can’t wait to give you a glimpse of the future of the in-car experience at one of our upcoming events.