Romy Müller
Faculty of Psychology
Chair of Engineering Psychology and Applied Cognitive Research
Technische Universität Dresden
01069 Dresden
Tel.: +49 351 463 35330
Fax: +49 351 463 37741
Visiting Address
Office Building Zellescher Weg (BZW)
Zellescher Weg 17
Room A225
Research Interests
My research focuses on two areas: the coordination of joint action, and human-machine interaction during fault diagnosis.
In my basic research on joint action, I am trying to find out how implicit signals about a partner's attention and action are processed and how they are used to guide a person's own action. In the past, I have asked how people interpret and use information about a remote partner's gaze in computer-mediated cooperation. Currently, I am interested in processes of action effect anticipation in joint tasks. If we interact with a partner and are able to predict how he will react to our actions, does this influence our action planning? I am extending research on ideomotor effect anticipation to the selection among competing task sets. That is, I am asking how task set selection processes are changed when knowing that a partner will either perform a compatible or incompatible task.
In my more applied research, I am investigating fault diagnosis and the role assistance systems can play in it. First, we study the cooperation between remote partners during fault diagnosis in experimental scenarios that are inspired by the process industries. Second, we study fault diagnosis in food manufacturing and ask how this activity can be supported by dialogue-based assistance systems (conversational case-based reasoning). The focus of our assistance concepts is to foster an understanding of causal relations. We do this both by conducting field studies in a company that produces and packages chocolate, and by conducting lab experiments that aim at transferring knowledge from cognitive psychology and Human Factors to dialogues between humans and assistance systems.
Several general questions are important to me in my applied research:
- How can we use knowledge from basic research to understand and enhance human-machine cooperation? For instance, what can the literature on decision making and cognitive control contribute to our understanding of operator challenges in modular chemical plants?
- How do the challenges that operators are faced with depend on the specifics of the technical systems, and how does this differ between domains? For instance, what distinguishes operator roles, tasks, and cognitive requirements between the process industries and discrete processing?
- How can we design assistance systems that do not simply tell operators what to do but aid them in understanding the situation? What do operators need to know about the causes of a fault and the interactions between the machine, process parameters and environmental factors?
Projects
An overview of the similiarities and differences between our current projects is provided here
- XAI-Dia: Explainable artificial intelligence for fault diagnosis: Impacts on human diagnostic processes and performance (DFG, 2022-2025)
- HyTec: Hypotheses during diagnostic problem solving in technical domains: Mental basis, process, and outcome (DFG, 2021-2024)
- KoMMDia: Cooperative human-machine dialogues for fault diagnosis in processing
machinery (BMBF, 2017-2020) - PlantCom: Supporting remote communication between control room and field in the process industries (DFG, 2016-2019)
- IdeJo-Action: Ideomotor effects in joint action: The influence of anticipated partner reactions on action control (DFG, 2014-2016)
- Communicative Eye Movements (PhD grant, 2008-2011)
Teaching
Lectures
-
Introduction to Cognitive Science / Cognitive Ergonomics
- Engineering Psychology
- Applied Cognitive Research
Seminars
- Human Factors Methods in Complex Work Domains
- Engineering Psychology and Applied Cognition
- Attention and Eye Movements
Other courses
Publications
Journal articles and book chapters
Schmidt, J., & Müller, R. (2023). Discipline differences in mental models: How mechanical engineers and automation engineers evaluate machine processes. Human Factors and Ergonomics in Manufacturing & Service Industries. http://doi.org/10.1002/hfm.21005
Müller, R., & Urbas, L. (2023). Model-based operator assistance: How to match engineering models with humans’ cognitive representations of their actions? In S. Mukherjee, V. Dutt, & N. Srinivasan (Eds.), Applied Cognitive Science and Technology: Implications of Interactions Between Human Cognition and Technology (pp. 73-88). Singapore: Springer. Link to book chapter
Hesse, P., Kaseler, E. L., Meier, J., Müller, R., & Abele, S. (2022). Logfilegestützte Erfassung und Spezifizierung des Kfz-Diagnoseprozesses von Fachleuten. In S. Schumann, S. Seeber, & S. Abele (Eds.), Digitale Transformation in der Berufsbildung: Konzepte, Befunde und Herausforderungen (Vol. 41, pp. 165-188). Bielefeld: wbv Media GmbH & Co. KG.
Müller, R., Kessler, F., Humphrey, D. W., & Rahm, J. (2021). Data in context: How digital transformation can support human reasoning in cyber-physical production systems. Future Internet, 13, 156. https://doi.org/10.3390/fi13060156
Bönsel, R., & Müller, R. (2021). Shared mental models for cooperative fault diagnosis: Effects on communication and diagnostic performance. International Journal of Human Factors and Ergonomics, 8(3), 215-244. https://doi.org/10.1504/IJHFE.2021.10038369
Müller, R., Paul, D., & Li, Y. (2021). Reformulation of symptom descriptions in dialogue systems for fault diagnosis: How to ask for clarification. International Journal of Human-Computer Studies, 145, 1-19. https://doi.org/10.1016/j.ijhcs.2020.102516
Dostert, J., & Müller, R. (2021). Motivational assistance system design for industrial production: From motivation theories to design strategies. Cognition, Technology & Work, 23(3), 507-535. https://doi.org/10.1007/s10111-020-00643-y
Hillig, S., & Müller, R. (2021). How do conversational case-based reasoning systems interact with their users: A literature review. Behaviour & Information Technology, 40 (14), 1544-1563. https://doi.org/10.1080/0144929X.2020.1767207
Müller, R., & Urbas, L. (2020). Adapt or Exchange: Making changes within or between contexts in a modular plant scenario. Journal of Dynamic Decision Making, 6, 1-20. https://doi.org/10.11588/jddm.2020.1.69326
Müller, R. (2020). How do partner reactions affect task set selection: Specific imitation or abstract tasks? Quarterly Journal of Experimental Psychology, 73(11), 2008-2025. https://doi.org/10.1177/1747021820937918
Müller, R., Gögel, C., & Bönsel, R. (2020). Data or interpretations: Impacts of information presentation strategies on diagnostic processes. Human Factors and Ergonomics in Manufacturing & Service Industries, 30(4), 266-281. https://doi.org/10.1002/hfm.20838
Müller, R., Paap, J. H., & Bönsel, R. (2020). Can users infer missing information in conversational case-based reasoning: The effects of providing context. International Journal of Human Factors and Ergonomics, 7(1), 55-79. https://doi.org/10.1504/IJHFE.2020.10029054
Müller, R., & Oehm, L. (2019). Process industries versus discrete processing: How system characteristics affect operator tasks. Cognition, Technology & Work, 21(2), 337-356. https://doi.org/10.1007/s10111-018-0511-1
Müller, R., Ullrich, L., Dessel, A.-K., Bengsch, A., Heinze, S., Müller, T., & Oehm, L. (2019). Case-based decision support for fault diagnosis: Do salient ratings discourage situation analysis? Human Factors and Ergonomics in Manufacturing & Service Industries, 29(1), 44-62. https://doi.org/10.1002/hfm.20757
Müller, R. (2019). Cognitive challenges of changeability: Adjustment to system changes and transfer of knowledge in modular chemical plants. Cognition, Technology & Work, 21(1), 113-131. https://doi.org/10.1007/s10111-018-0489-8
Müller, R., & Jung, M. L. (2018). Partner reactions and task set selection: Compatibility is more beneficial in the stronger task. Acta Psychologica, 185, 188-202. https://doi.org/10.1016/j.actpsy.2018.02.012
Müller, R., & Urbas, L. (2017). Cognitive challenges of changeability: Multi-level flexibility for operating a modular chemical plant. Chemie Ingenieur Technik, 89(11), 1409-1420. https://doi.org/10.1002/cite.201700029
Müller, R., Narciss, S., & Urbas, L. (2017). Interfacing Cyber-Physical Systems with Human Decision Makers. In H. Song, D. B. Rawat, S. Jeschke & C. Brecher (Eds.), Cyber-Physical Systems: Foundations, Principles and Applications (pp. 145-160). Elsevier.
Müller, R. (2016). Does the anticipation of another person’s compatible reactions facilitate action planning in joint tasks? Psychological Research, 80(4), 464-486. https://doi.org/10.1007/s00426-015-0670-0
Müller, R., Helmert, J. R., & Pannasch, S. (2014). Limitations of gaze transfer: Without visual context, eye movements do not to help to coordinate joint action, whereas mouse movements do. Acta Psychologica, 152(1), 19-28. https://doi.org/10.1016/j.actpsy.2014.07.013
Müller, R., Helmert, J. R., Pannasch, S., & Velichkovsky, B. M. (2013). Gaze transfer in remote cooperation: Is it always helpful to see what your partner is attending to? Quarterly Journal of Experimental Psychology, 66(7), 1302-1316. https://doi.org/10.1080/17470218.2012.737813
Müller, R. (2012). Blickbewegungen in der computermediierten Kooperation: Chancen und Schwierigkeiten in der Verwendung einer weniger expliziten Kommunikationstechnologie. Unpublished dissertation. Dresden: Technische Universität Dresden. Download Dissertation
Pannasch, S., Helmert, J.R., Müller, R. & Velichkovsky, B.M. (2012). The analysis of eye movements in the context of cognitive technical systems: Three critical issues. In A. Esposito, M. A. Esposito, A. Vinciarelli, R. Hoffmann, & V. C. Müller (Eds.) Cognitive Behavioural Systems, LNCS (Vol. 7403, pp. 19-34) Berlin: Springer-Verlag. Link to article
Conference presentations: Human-machine interaction
Müller, R., Mangold, A., Ott, F., & Schmidt, S. (2023). Overtrust in explainable AI that is only partly correct: Do people search for non-highlighted faults during visual quality control? Talk at the 13. Fachgruppentagung der AOW- und Ingenieurspsychologie 2023. Kassel, Germany, September 12-15.
Schmidt, J., & Müller, R. (2023). Diagnosing faults in different technical systems: How domain characteristics can be used to predict requirements for diagnosticians. Talk at the 13. Fachgruppentagung der AOW- und Ingenieurspsychologie. Kassel, Germany, September 12-15.
Müller, R., Reindel, D.F., & Stadtfeld, Y.D. (2023). How does correct and incorrect explainable artificial intelligence affect visual search? Poster presented at the 65. Tagung experimentell arbeitender Psychologen. Trier, Germany, March 26-29.
Schmidt, J., Kaseler, E. L., Wollermann, L., & Müller, R. (2023). Can people better express their understanding of system functioning graphically or verbally? Poster presented at the 65. Tagung experimentell arbeitender Psychologen. Trier, Germany, March 26-29.
Müller, R. (2022). Herausforderungen der Mensch-Maschine-Interaktion: Das Dilemma zwischen leicht bedienbaren Systemen und Übervertrauen in Technik. Talk at the 15th Annual Meeting of the DGFT Deutsche Gesellschaft für Feinwerktechnik, Dresden, October 13-14.
Hesse, P., Kaseler, E. L., Müller, R., & Abele, S. (2022). Logfile- und blickdatenbasierte Erfassung des Diagnoseprozesses von Experten in einer Kfz-Computersimulation. Talk at the 7th Journal of Technical Education (JOTED) Technology Didactics Symposium and BPT 10th Anniversary Celebration. Stuttgart, Germany, November 23-25.
Kaseler, E. L., Müller, R., Schmidt, J., Hesse, P., & Abele, S. (2022). Functional understanding of car mechatronic trainees and its relation to fault diagnosis performance. Talk at the 11th EARLI SIG14 Conference 2022, Paderborn, Germany, August 17-19.
Hesse, P., Kaseler, L., Müller, R., Abele, S. (2021). Logfile- und blickdatenbasierte Erfassung des Diagnoseprozesses von Experten in einer Kfz- Computersimulation. Talk at the Sektionstagung Berufs- und Wirtschaftspädagogik. Bamberg, Germany, September 15-17.
Müller, R. (2020). Was unterscheidet Schokolade von Flugzeugen und Handys: Domänenspezifität in der Mensch-Maschine-Kooperation. Keynote at the BMBF conference "Technikbasierte Dienstleistungssysteme", Frankfurt, July 09.
Müller, R. (2020). Why psychologists should know about chocolate: A domain-centred perspective on human-machine cooperation. Talk at the TU Kaiserslautern Cognitive Science Colloquium, Kaiserslautern, January 16.
Müller, R. (2019). Mensch-Maschine-Interaktion in Produktionsanlagen: Sollten Nutzerschnittstellen es dem Menschen einfach machen? Talk at the 20th Anniversary of VDMA Software und Digitalisierung, Würzburg, November 14.
Jaster, S., & Müller, R. (2019). Conception of interfaces to represent functional relations in a packaging machine. Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Nantes, France, October 2-4.
Koshman, N., Blümel, R., Bönsel, R., & Müller, R. (2019). Machine effectiveness made understandable: Integrated displays that inform operators about effectiveness components and possible interventions. Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Nantes, France, October 2-4.
Mesletzky, L., & Müller, R. (2019). Eliciting strategies for diagnosing faults of packaging machines: A scenario-based study with maintenance technicians. Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Nantes, France, October 2-4.
Müller, R. & Urbas, L. (2019). Adapt or Exchange: Supporting decisions in modular plants with process relations or cases. Poster presented at the EuroCogSci 2019, Bochum, September 2-4.
Gögel, C., Bönsel, R., & Müller, R. (2019). Data or interpretations: The consequences of presenting different types of information in assistance systems for fault diagnosis. Poster presented at the 38th Meeting of the European Group of Process Tracing Studies, Dresden, June 12-14.
Blunk, A., Bönsel, R., & Müller, R. (2019). Understanding question intent in dialogue systems: The impact of explanations and clarifications on user behavior and confidence. Poster presented at the 61. Tagung experimentell arbeitender Psychologen, London, April 14-17.
Müller, R. (2019). Assistenzsysteme für komplexe Produktionsanlagen: Wie viel Information braucht der Bediener? Talk at VDMA Mitgliederversammlung NuV, Dresden, April 11.
Müller, R. (2018). Assistenzsysteme für komplexe Produktionsanlagen: Wie viel Information braucht der Bediener? Talk at Packnet 12/2018, Dresden, December 13.
Gögel, C., Bönsel, R., & Müller, R. (2018). Operator strategies in the food processing industry: Individual differences in handling machine faults. Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Berlin, October 8-10.
Bönsel, R., Schegner, L., Heinze, S., & Müller, R. (2018). Supporting collaborative fault diagnosis: Complete information transfer or strategic hypothesis testing? Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Berlin, October 8-10.
Rahm, J., Graube, M., Müller, R., Klaeger, T., Schegner, L., Schult, A., Bönsel, R., Carsch, S., Oehm, L., & Urbas, L. (2018). KoMMDia: Dialogue-driven assistance system for fault diagnosis and correction in cyber-physical production systems. In Proceedings of the 23rd International Conference on Emerging Technologies and Factory Automation, Torino, September 4-7.
Müller, R. & Paul, D. (2018). Common ground with dialogue systems for fault diagnosis: How should they ask for clarification? Poster presented at the KogWis 2018, Darmstadt, September 3-6.
Müller, R. (2018). Human-machine cooperation in highly automated systems. Talk at the ARC Industry Forum Europe, Barcelona (Sitges), May 15-16.
Bönsel, R., Schegner, L., Heinze, S., Urbas, L., & Müller, R. (2018). The impact of shared mental models on team performance in a fault diagnosis task. Poster presented at the 60. Tagung experimentell arbeitender Psychologen, Marburg, March 11-14.
Müller, R., Dessel, A.-K., Bengsch, A., Heinze, S., & Oehm, L. (2017). Case-based decision support for fault diagnosis: Salient outcome ratings do not discourage thorough situation analysis. Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Rome, September 28-30.
Müller, R. (2017). Mensch-Maschine-Kooperation in hochautomatisierten Systemen. Talk at the VVD-Anwenderforum, Dresden, September 26.
Oehm, L., Müller, T., Müller, R., Ziegler, J., Majschak, J.-P., Urbas, L., & Schult, A. (2016). Kooperative Störungsdiagnose durch Bediener und Assistenzsystem für Verarbeitungsanlagen. Poster presented at the Zweite Transdisziplinäre Konferenz: Technische Unterstützungssysteme, die die Menschen wirklich wollen. Hamburg, December 12-13.
Müller, R., Ullrich, L., Müller, T., Ziegler, J., & Oehm, L. (2016). Case-based decision support for fault diagnosis: The impact of salient case ratings on situation analysis and solution focus. Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Prague, October 26-28.
Riese, L., Dostert, J., Hoffmann, R., Pfeffer, J., Urbas, L, & Müller, R. (2016). Communication between control room and field operators in the process industries. Poster presented at the Annual Meeting of the Human Factors and Ergonomics Society, Prague, October 26-28.
Müller, R. (2015). Der Mensch auf Standby: Wie Automatisierung unser Denken und Handeln beeinflusst. Talk at the Gesellschaft für Arbeitswissenschaft Herbstkonferenz, Dresden, September 21-22.
Lambeck, C., Fohrholz, C., Leyh, C., Supulniece, I., & Müller, R. (2014). Commonalities; Contrasts: An Investigation of ERP Usability in a Comparative User Study. Proceedings of the 22nd European Conference on Information Systems. Tel Aviv, Israel.
Lambeck, C., Müller, R., Fohrholz, C., & Leyh, C. (2014). (Re-) Evaluating User Interface Aspects in ERP Systems - An Empirical User Study. Proceedings of the 47th Hawaiian International Conference on System Sciences. Waikoloa, Hawaii, USA.
Conference presentations: Joint action and gaze transfer
Müller, R. (2019). Partner reactions affect task set selection: The roles of specific imitation and abstract task set compatibility. Talk at the 61. Tagung experimentell arbeitender Psychologen, London, April 14-17.
Müller, R., Wienß, E., & Bärwinkel, H. (2018). Does task asymmetry determine the influence of anticipated partner reactions on task set selection? Poster presented at the 60. Tagung experimentell arbeitender Psychologen, Marburg, March 11-14.
Müller, R. (2017). Anticipated partner reactions do not affect grasp initiation in joint object manipulation. Poster presented at the 59. Tagung experimentell arbeitender Psychologen, Dresden, March 27-29.
Müller, R. (2017). Coherence judgements in the Remote Associates Task are faster when people believe that another human has seen the triad before. Talk at the 59. Tagung experimentell arbeitender Psychologen, Dresden, March 27-29.
Müller, R. & Jung, M.L. (2016). Incompatible partner reactions improve performance in the weaker task of a picture-word interference paradigm – But only if they can be ignored. Talk at the 58. Tagung experimentell arbeitender Psychologen, Heidelberg, March 21-23.
Müller, R. (2015). Visualisierung von Blickübertragung: Cursors, Highlights und Spotlights. Talk at the Ebiss Workshop, Dortmund, December 03-04.
Müller, R. (2015). Temporally compatible partner reactions facilitate action initiation and produce contrast effects on movement duration. Talk at the EuroAsianPacific Joint Conference on Cognitive Science, Torino, September 25-27.
Müller, R. (2015). Joint spatial and temporal response-effect compatibility: Do anticipated reactions of a partner facilitate the planning and execution of hand movements? Poster presented at the 6th Joint Action Meeting, Budapest, July 1-4.
Müller, R. (2015). The impact of anticipated partner reactions on attentional selection. Talk at the 57. Tagung experimentell arbeitender Psychologen, Hildesheim, March 8-11.
Müller, R. (2014). The effects of observing other people’s gaze: Faster intuitive judgments of semantic coherence. Poster presented at the KogWis 2014, Tübingen, September 29 - October 2. Download short paper
Müller, R., & Kammer, D. (2013). Does the anticipation of a partner's reaction affect action planning? Spatial action-effect compatibility in a joint task. Poster presented at the CogSci 2013, Berlin, July 31 - August 3.
Müller, R. (2013). Response-effect compatibility with self-generated and partner-produced effects. Poster presented at the 55. Tagung experimentell arbeitender Psychologen, Vienna, March 25-27.
Müller, R., Helmert, J.R., Pannasch, S., & Velichkovsky B.M. (2011). Following closely? The effects of viewing conditions on gaze versus mouse transfer in remote cooperation. Paper presented at the Dual Eyetracking Workshop 2011, Aarhus, September 24-28. Download full paper
Müller, R., Helmert, J.R., Pannasch, S., & Velichkovsky B.M. (2011). In search for the optimal communication means in joint cooperation: Gaze versus mouse transfer. Poster presented at the 16th European Conference on Eye Movements, Marseille, August 21-25.
Müller, R., Helmert, J.R., Pannasch, S., & Velichkovsky B.M. (2010). How does the level of interaction mediate the benefit of gaze transfer in a cooperative puzzle task? Talk at the KogWis 2010. Potsdam, October 3-6.
Müller, R. (2010). Gaze transfer in computer-mediated cooperation. Talk at the Fourth International Conference on Cognitive Science, Tomsk, June 21-26.
Müller, R., Reiche, A., Pannasch, S. & Velichkovsky B.M. (2010). Communication by way of gaze transfer: Using eye-movement data to improve computer-mediated cooperation in a puzzle solving task. Poster presented at the 52.Tagung experimentell arbeitender Psychologen, Saarbrücken, March 22-24.
Müller, R., Pannasch, S. & Velichkovsky B.M. (2009). Comparing eye movements for perception and communication: Changes in visual fixation durations and saccadic amplitudes. Perception 38 ECVP Abstract (suppl), p. 23.
Müller, R., Pannasch, S. & Velichkovsky B.M. (2009). Gaze-based communication influences visual fixation durations and saccadic amplitudes. Poster presented at the 51. Tagung experimentell arbeitender Psychologen, Jena, March 29 - April 1.
Conference presentations on other topics
Müller, R. (2019). Warum die Eisbären uns nicht motivieren, gegen den Klimawandel vorzugehen: Psychologie und Umweltschutz. Talk at the 25th Anniversity of the Dresdener Seniorenakademie, Dresden, November 05.
Müller, R. (2018). Psychologie und Klimaschutz. Talk at the 23. Tag der Erneuerbaren Energien, Oederan, April 28.
Kammer, D., Müller, R., Wojdziak, J., Zavesky, M., Weyprecht, P., Schönefeld, F., & Groh, R. (2012). Innovationspotenzial validieren: Anwendungsgebiete für die 3D-Plattform Bildsprache LiveLab. Innovationsforum open4INNOVATION2012 regional kooperativ-global innovativ Beiträge zum Fachforum. Technische Universität Dresden, Germany.
Helmert, J.R., Müller, R., Pannasch, S., & Velichkovsky, B.M. (2008). Influence of scene background and spatial arrangement of objects on gaze behaviour. Perception 37 ECVP Abstract (suppl), p. 95.