Menu
What’s missing? Depth! The variable depth between the driver and, say, 30 meters in front of the car is vital to making a real impact on safety.
Volumetric HUDs use augmented reality to project flat information in front of the driver’s face and display virtual overlays such as lane markers or GPS in real-time. A tiny flat left-turn arrow is handy, but your phone or touch-screen center display can already do it if you look down or askew. A 3D arrow that curves in front of you to show exactly where the next intersection is in real life is much more convenient.
This article will address the closed-loop approach to virtual AR-HUD development and examine the key benefits that automotive software engineers can get from it.
As cars become smarter and more autonomous, head-up displays are gaining more attention from automakers and consumers.
HUDs work by projecting an inverted image onto the windshield, which is then reflected directly into the driver’s eyes. Cars fitted with HUDs often have an odd rectangular shape at the top of the dashboard in front of the dials; that’s where the image is actually projected from.
Displaying informative graphics on the windshield helps drivers maximize safety and keep their eyes on the road without getting distracted by looking at the dashboard. But just as importantly, HUDs create confidence that ADAS solutions such as cruise control and lane-keeping assist system are working properly.
Since the driver’s eyes must be able to scan the real world and not the dashboard or smartphone screen, a combination of AR, sensors such as RADAR and LIDAR, machine learning, and cameras deems an effective solution to this challenge. It provides a good opportunity to raise driver awareness of the dangers ahead: kids on bikes, potholes, and so on.
HUD, which first appeared in the aerospace industry, is now a standard feature on some luxury vehicles and will gradually be introduced to other models and classes. Transparency Market Research estimates that the global automotive HUD market will exceed $4.7 billion by the end of 2030, reflecting an annual growth rate of 22%.
The biggest advantage of AR-HUD is safety. Even the slightest glance away from the road can lead to a loss of concentration and possibly an accident. By putting all the information you need while driving in the driver’s field of vision, AR-HUD ensures that you always keep your eyes on the road.
And since this information is overlaid on top of the physical road, you don’t need to translate the information provided by the mobile or embedded dashboard screen into the actual road. For example, a complex intersection may include several left turns. By overlaying directions on the road itself, AR-HUD lets you instantly see which turn to take.
By displaying important information such as speed and navigation data on the windshield, the HUD minimizes the need for drivers to get distracted to check the road.
As advanced driver assistance systems (ADAS) take over the global automotive market, HUDs are doing another vital job – they help increase people’s confidence in these systems.
Suppose a driver relies on ADAS’s lane-keeping capabilities. In that case, their confidence can be increased by actually seeing the lane markings plotted graphically on the HUD and watching the vehicle stay within those markings on its own. Similarly, a HUD can be designed to graphically highlight an object in the road ahead, such as a vehicle or pedestrian, allowing these objects to be recognized and monitored by ADAS sensors. HUDs can help build confidence in machine perception systems by providing a graphical representation of what the sensors are “seeing” as the car moves on the road.
Overlaying smart graphics on a head-up display, instead of simply displaying real-time operational data collected from the vehicle, means introducing AR technologies. Embedded AR software solutions are needed to collect data from ADAS sensors, develop hierarchical control logic, and graphically define both menu screens and dynamic display elements. Innovative human-machine interfaces can be programmed to display a variety of graphics and animations that help drivers navigate the road and increase their confidence and trust in their vehicle’s ADAS capabilities.
AR-HUDs must be carefully designed and tested because they are an essential component of driving safety. They must display accurate images with minimal latency through a user interface that keeps drivers focused on the road as much as possible while sharing all the real-time information needed to ensure maximum road safety. At the same time, they should not overload the driver’s cognitive system by displaying too much information.
They must be designed to work effectively in a wide variety of traffic scenarios, including “edge cases” that can confuse one of the vehicle’s sensors, such as a pedestrian crossing in foggy conditions.
Augmented reality arrows are projected onto the road ahead, providing precise navigation, so you never miss a turn.
Each time pedestrians are crossing the road, the driver gets a visual notification.
The driver gets a notification in case of potential traffic lane violations.
Lane-level hints ensure you are in the right lane at the right time.
The display reminds the driver to keep a distance from the vehicle in front.
Surrounding vehicles are accurately tagged in poor driving conditions, helping the driver to get to their destination safely.
While the market for augmented reality is booming, launching AR-based HUD projects quickly, affordably, and reliably is a challenge for engineering teams from both automakers and Tier 1 suppliers. These displays combine advanced optics, hardware, firmware, and human-machine interfaces
in a single product system that must perform reliably in every possible driving scenario. This is due to the high technical complexity and because the product development process includes many functions and stakeholders from the entire engineering organization.
Since an AR-HUD combines optics, hardware, software, and human-machine interfaces (HMIs), building multiple physical prototypes is costly and time-consuming.
Automotive OEMs and their Tier 1 suppliers are challenged to assemble interdisciplinary HUD development teams to build cutting-edge optical and software capabilities, optimize user experience, meet stringent security and validation requirements, and quickly launch their designs to meet growing market demand – all at the same time. As the technology still evolves, HUD development relies heavily on the R&D capabilities to future-proof design, IoT-data collection, and overall efficiency and quality.
An integrated, closed-loop development approach proves effective for AR-based HUD solutions. Instead of relying on different technology tools, disparate processes, and data, engineers can collaborate more closely with shared datasets, seamless switching, and a unified HUD development workflow. Implementing an integrated HUD development process requires deep domain knowledge, an available pool of software and hardware developers, a high level of process automation, overall maturity, and access to robust R&D resources. That’s a big challenge for OEMs and Tier 1 suppliers.
However, technology partnerships with specialized AR-IoT, and digital twin development companies like rinf.tech allow all technical components to be designed and tested in the closed-loop digital environment while using an R&D Center for idea validation and pilot solutions.
The closed-loop design workflow begins with the use of optical design software to design the optical elements of the head-up display. Working from a windshield surface design provided by an automotive OEM or Tier 1 supplier, a team of optical engineers develops the HUD visual elements.
Using CAD-agnostic design tools enables automotive software engineers to check the feasibility and practicality of the desired optics design, given constraints such as windshield material and shape. It helps dev teams better understand how the HUD and its performance will be affected by optical phenomena such as distortion or ghost images and physical phenomena such as thermal effects that can degrade materials over time.
With built-in CAD analysis, we at rinf.tech can automatically check and validate the vehicle maker’s requirements for compliance and performance against a given optical design. Because any visible or physical issues are identified early in product development, rapid iterations that quickly lead to the final optimized optic design are possible.
Once the optical display capabilities for the HUD are complete, embedded software engineering teams start working on a seamless and user-friendly HUD design.
Using sensor-agnostic tools, they collect data from the existing in-vehicle sensor system and then use AR graphics to help the driver visualize that data in real-time. By sending alerts and warnings, the software creates a bridge between the vehicle’s sensors and the HUD’s optics, enabling faster decision-making in the event of unforeseen circumstances. Overlay graphics, directional arrows, lighting, and other AR components draw the driver’s attention to critical objects or data points displayed on the windshield.
Use cases for augmented reality design capabilities include:
All human-machine interfaces are tested to evaluate all HUD controls’ ease of use and responsiveness combined with complete digital cockpit simulation. Hand movements and finger touches can be simulated with a digital twin technology to optimize cockpit interactions and HUD design. The outcome is a genuinely user-centric design that upholds both safety and the automaker’s unique brand.
A closed-loop approach ensures that HUD designers consider user experience upfront rather than retroactively after investing a certain amount in physical prototyping and reengineering.
Compared to traditional HUD development approaches characterized by siloed processes, manual handoffs, and reliance on physical testing, the virtual process supported by rinf.tech has a number of distinct advantages, including:
A closed, integrated approach dramatically simplifies and speeds up design iterations and functional transfer.
By eliminating manual steps and physical prototypes, virtual development saves significant costs. In addition, late-stage reengineering is eliminated because any engineering problems are flagged and fixed early in development.
Instead of relying on disparate technology tools, disparate processes, and disparate data, engineers can collaborate more closely with an integrated technology platform, shared dataset, seamless switching, and a unified HUD development workflow.
Physical prototyping and HUD testing are expensive and pose data security risks. By testing and validating their HUD designs in the virtual space, engineers can be creative, take risks, and explore innovations that lead to market leadership without incurring high costs or compromising human safety.
Today, UX is becoming a primary competitive advantage in the automotive market. AR-based HUD development makes it easy for automakers and Tier 1 suppliers to achieve world-class user experiences that enhance their brand image.
Instead of relying on guestimations, the entire HUD product development team can work from a shared perspective backed by data. As improvements are made to the display design, all stakeholders in each feature can see quantifiable results.
Because HUDs are a critical component of the autonomous vehicle, the security stakes are high. Virtual testing and validation ensure every component meets safety standards and performs as expected in real-world scenarios, from sensors and software to optics and controls. The test coverage includes edge cases that can confuse some sensors, such as sun glare or an oddly shaped object such as a pedestrian in a wheelchair.
As the global market for autonomous vehicles and ADAS grows, the demand for AR-HUDs will only increase.
Engineers need a fast, accurate, and cost-effective way to design, optimize, test, and validate volumetric HUDs to meet this demand. Managing functional specifications, quality targets, human factors, and safety criteria must be faster and more accurate than ever without compromising reliable performance across thousands of potential operating parameters.
Using cutting-edge software tools and technologies such as digital twin, HUD product development teams can implement the revolutionary innovations needed to achieve market leadership and profitability, speed to market, and product confidence.
