Erasmus Mundus Industrial Challenge Project (Summer Semester 2023)
by Prof. Steve Goddard, University of Iowa.
Course Credits: 3
Dates: 1st June - 5th July 2023
Location: Lab @ Chair of RTS (in person)
Enroll here to get further information and updates.
Automotive head-up displays (HUDs) provide basic driving information (e.g., speed or route guidance) projected on a small display located 2-3 meters in front of the driver. Future automotive HUDs, however, will provide much more useful information to the driver using augmented reality (AR) technology. That is, graphical information will be projected onto the automobile's windshield in the driver's field of view such that the graphic appears fused to the real-world object seen by the driver. An example of this concept is shown in Figure 1. Notice the bicycle icon projected on the bicycle rider and the bridge height displayed for the upcoming bridge, in addition to more traditional navigational information
Augmented-Reality (AR) Head Up Display (HUD) example showing graphics fused to objects seen in the real-world by the driver.
To make this concept a reality:
- Images must be projected 10-20 m in front of the driver, with a minimum distance of 7 m.
- Sensors must be sampled and processed at 120 Hz, including object localization and recognition, velocity vectors computed, driver's head poses computation, and objects projected on the windshield with less than 50 ms latency.
Companies such as Texas Instruments are developing digital light processing projectors that can project on a holographic film in the windshield using a digital micromirror device[1], which will address the first challenge.
The second challenge is the focus of this class. Researchers have developed programs that can sample the environment, detect and localize objects, compute head-poses for the driver, and project relevant objects on a camera image, which is equivalent to projecting the objects onto a windshield-infused holographic film.
Such applications have stringent real-time and safety requirements and are executed on a heterogeneous System-on-Chip (SoC) hardware architecture with a non-deterministic execution model of the involved computation elements and the co-executing applications. This leads to contention in the shared hardware resources and makes it challenging to design and orchestrate the applications. As of result of such limitations, HUD update rates are often in the range of 10-60 Hz, with latency exceeding 1 second. Thus, there is a growing demand for novel methodologies, tools, and best practices to assist application designers working on such high-performance real-time systems.
Engineers from ARM have identified the following challenges [2]:
- Analysis: Perform the response time analysis and worst-case execution time analysis following any methodology deemed suitable to the use case and platform considered. For this step, the challenge participants can choose to assume the absence of shared resources and other observable interference effects. The input for this first step is a description of the software tasks and their dependencies, commonly expressed as direct acyclic graphs (DAGs).
- Develop techniques or tools to identify the execution frequency, duration, and resource usage of each node in each DAG.
- Automate this analysis.
- Optimizations:
- Scheduling: Design one or more scheduling policies that can achieve better system utilization and data sharing between tasks.
- Create a static schedule for this application.
- Does static scheduling result in better performance (reduced latency, increased resource utilization, higher processing rates, and more deterministic execution than dynamic scheduling?
- Evaluate dynamic scheduling approaches, adapting existing approaches or develop new algorithms, for the AR-HUD tasks.
- Why is static scheduling better/worse than dynamic scheduling?
- Resource mapping: Efficient mapping of tasks to the various hardware components (processing nodes and resources) in the platform, in order to maximize efficiency and minimize contention.
- Shared resource interference monitoring and performance isolation: Propose and/or implement shared resources monitoring strategies and design hardware and/or software techniques for shared resource contention mitigation.
- Scheduling: Design one or more scheduling policies that can achieve better system utilization and data sharing between tasks.
In this class, working as individuals or in teams, we will:
- Study the application and real-time processing challenges.
- Conduct a literature review regarding possible solutions to identified challenges.
- Evaluate and/or implement possible solutions from the literature.
- Develop new techniques that address open problems.
Note: There are limited number of seats for this course and will be assigned on first come first serve basis.
Enroll here to get further information and updates.
References
- M. Firth, “Introduction to automotive augmented reality head-up displays using ti dlp® technology,” Technical doc- ument, May, 2019.
- M. Andreozzi, G. Gabrielli, B. Venu, and G. Travaglini, “Industrial Challenge 2022: A High-Performance Real-Time Case Study on Arm,” in 34th Euromicro Conference on Real-Time Systems (ECRTS 2022), ser. Leibniz International Proceedings in Informatics (LIPIcs), M. Maggio, Ed., vol. 231. Dagstuhl, Germany: Schloss Dagstuhl – Leibniz- Zentrum für Informatik, 2022, pp. 1:1–1:15. [Online]. Available: https://drops.dagstuhl.de/opus/volltexte/2022/16318
- ECRTS ARM Industrial Challenge: A High-Performance Real-Time Case Study on Arm, last accessed 25th Jan 2023, link: https://www.ecrts.org/arm-industrial-challenge/
Information
More information can be found in KIS.