[MA] Improving Explainability and Trust in Cobotic Workflows via Multimodal Mixed Reality

The rapid development of mixed reality (MR) technologies brings higher processing power and opens broader interaction opportunities, enabling better integration into human-robot collaboration within shared environments. Visualizing the raw sensor data no longer suffices. An explainable multimodal MR approach is essential to improve this integration, helping to reduce the cognitive load experienced by the user. Similar work has been done before, but it was very limited by the capabilities of the TurtleBot3. This continuation work explores the potential of utilizing Max/Moritz, a more capable robot, in explainable MR scenarios that combine several interaction modalities such as visual (camera streams), physical tracking (gaze, head and hand positions, gestures), and auditory interactions (voice commands and responses) for enhanced transparency, trust, and error diagnosis/prognosis.

Betreuer: Victor Victor