Oct 20, 2025
Safe use of large language models in oncology: Researchers from EKFZ for Digital Health contribute to new international guidelines
Under the leadership of Prof. Jakob N. Kather, Professor of Clinical Artificial Intelligence at EKFZ for Digital Health, TUD, and Dresden University Hospital Carl Gustav Carus (NCT/UCC), an international expert panel has developed guidelines for the safe use of large language models (LLMs) in oncology. Prof. Stephen Gilbert and Dr. med. Isabella Wiest, EKFZ for Digital Health, also contributed their expertise. The initiative was convened under the European Society for Medical Oncology (ESMO) as part of its “Real World Data & Digital Health Task Force”. The new framework aims to help patients, clinicians and institutions to adopt AI language tools responsibly.
The European Society for Medical Oncology (ESMO) has just released the ESMO Guidance on the Use of Large Language Models in Clinical Practice (ELCAP), the first structured set of recommendations to bring AI language models into oncology safely and effectively. ELCAP has been published in ESMO’s peer-reviewed journal Annals of Oncology, coinciding with a session on “ChatGPT and cancer care” at the ESMO Congress 2025 in Berlin.
ELCAP focuses on assistive LLMs that operate under human oversight, supporting clinicians by providing information or drafting content rather than taking independent actions. “These systems are designed to enhance – and not replace – clinical workflows and decision-making,” said Jakob N. Kather, Deputy Chair of the ESMO Real World Data & Digital Health Task Force, Professor of Clinical AI at TUD and co-author of the study. “At the same time, the guidance acknowledges the rapid emergence of autonomous, or ‘agentic’, AI models capable of initiating actions without direct prompts, which raises distinct safety, regulatory and ethical challenges and will require dedicated future guidance,” he added.
“One of the main challenges in bringing LLMs into clinical practice is navigating regulatory constraints and the heterogeneous regulatory implementation across regions – even in the EU. We need EU-wide pragmatic standards, governance that balances the essential safeguards for transparency, validation, and human oversight while enabling the flexibility needed for local implementation and use,” explained Stephen Gilbert, Professor of Medical Device Regulatory Science at EKFZ for Digital Health, TUD.
Across settings, the reliability of outputs depends on the completeness and correctness of input data: gaps in clinical documentation or partial patient queries can lead to inaccurate or misleading responses, underscoring the need for supervision and clear escalation routes.
“ELCAP recognises that the value of language models depends on who is using them,” said Miriam Koopman, Chair of the ESMO Real World Data & Digital Health Task Force, co-author of the paper. “By distinguishing patient‑facing, clinician‑facing and background institutional systems, we set expectations for each context: supervised pathways for patients, validated and transparent tools for clinicians, and continuously monitored, well-governed systems embedded in the electronic health records.”
As the use of LLMs accelerates across oncology, ELCAP recognizes that opportunities and risks vary depending on the user – whether patients, clinicians or institutions, and therefore anchors the recommendations in a three‑type structure, translating high‑level principles into 23 consensus statements for day‑to‑day practice:
- The first category (Type 1) addresses patient‑facing applications such as chatbots for education and symptom support, which should complement clinical care and operate within supervised pathways with explicit escalation and robust data protection.
- The second category (Type 2) covers healthcare professional‑facing tools such as decision support, documentation and translation, which require formal validation, transparent limitations, and explicit human accountability for clinical decisions.
- The third category (Type 3) concerns background institutional systems integrated with electronic health records for tasks like data extraction, automated summaries and clinical‑trial matching; these systems require pre‑deployment testing, continuous monitoring for bias and performance change, institutional governance and re‑validation when processes or data sources change.
“ESMO’s priority is to ensure that innovation translates into measurable benefit for patients and workable solutions for clinicians. With ELCAP, ESMO provides a pragmatic, oncology‑specific framework that embraces AI while upholding clinical responsibility, transparency and robust data protection,” said Fabrice André, ESMO President.
ELCAP was developed between November 2024 and February 2025 by a 20‑member international panel spanning oncology, AI, biostatistics, digital health, ethics and the patient perspective, convened under the ESMO Real World Data & Digital Health Task Force.
Publication
E.Y.T. Wong, L. Verlingue, M. Aldea, M.A. Franzoi, R. Umeton, S. Halabi, N. Harbeck, A. Indini, A. Prelaj, E. Romano, E.Smyth, I. B. Tan, A. Valachis, J. Vibert, I. C. Wiest, Y.H. Yang, S. Gilbert, G. Kapetanakis, G. Pentheroudakis, M. Koopman, J.N. Kather: ESMO guidance on the use of Large Language Models in Clinical Practice (ELCAP); Annals of Oncology, 2025.
The full publication is available at: https://www.annalsofoncology.org/article/%20S0923-7534(25)04698-8/fulltext
DOI: 10.1016/j.annonc.2025.09.001
Contact:
Anja Stübner | Dr. Viktoria Bosak
Else Kröner Fresenius Center (EKFZ) for Digital Health
Faculty of Medicine Carl Gustav Carus
TUD Dresden University of Technology
Else Kröner Fresenius Center (EKFZ) for Digital Health
The EKFZ for Digital Health at TUD Dresden University of Technology and University Hospital Carl Gustav Carus Dresden was established in September 2019. It receives funding of around 40 million euros from the Else Kröner Fresenius Foundation for a period of ten years. The center focuses its research activities on innovative, medical and digital technologies at the direct interface with patients. The aim here is to fully exploit the potential of digitalization in medicine to significantly and sustainably improve healthcare, medical research and clinical practice. digitalhealth.tu-dresden.de
The European Society for Medical Oncology (ESMO)
Representing more than 45,000 oncology professionals, ESMO is a reference for oncology education and information. Driven by a shared determination to secure the best possible outcomes for patients, ESMO is committed to standing by those who care about cancer through addressing the diverse needs of #ONEoncologycommunity, offering #educationforLIFE, and advocating for #accessiblecancerCARE.
www.esmo.org