Ten German organizations from business, science and media are developing the European answer to GPT-3^*, including the Center for Information Services and High Performance Computing (ZIH) at TU Dresden. The German Federal Ministry for Economic Affairs and Climate Action is funding the Open GPT-X project within the Gaia-X funding inititative with around 15 million euros.

Under the leadership of the Fraunhofer Institutes for Intelligent Analysis and Information Systems (IAIS) and for Integrated Circuits (IIS), the OpenGPT-X project is starting with the goal of developing a large AI language model for Europe. Particular attention is being paid to data protection as well as European language diversity.

"International competitors have already recognized the enormous disruptive potential of AI language technologies for business, industry and society. A European AI language model like OpenGPT-X is therefore imperative to ensure Europe's digital sovereignty and market independence," says Dr. Nicolas Flores-Herr, head of the project at Fraunhofer IAIS.

Due to the high technical requirements, such as computing power, such powerful language models can so far only be implemented by large companies or consortia. Many well-known partners could be won for this project.

The ZIH of TU Dresden and the Forschungszentrum Jülich will provide high-performance computing capacities and work on performance improvements of models and hardware usage, whereas Fraunhofer IAIS will develop the AI models together with DFKI and the companies Aleph Alpha and Alexander Thamm GmbH. The integration into the Gaia-X structure will be managed by 1&1 IONOS SE. ControlExpert and the WDR, which are developing a use case in the project, will contribute domain-specific data for the development of the models, as well as other partners. The communication of the project is the responsibility of the KI-Bundesverband.

"ZIH is involved at different levels of the project: In addition to the HPC resources we are providing for the project, we will investigate various aspects of the performance of the language models. In addition to the quality of the predictions of such models, aspects such as parallel efficiency and energy consumption during the training of the models are becoming of growing importance. Especially in the case of the large language models targeted in the project, there are high potential savings. Finally, we are concerned with the connection and expansion of the project's internal data management," specifies Dr. René Jäkel, research associate at ZIH and managing director of ScaDS.AI Dresden/Leipzig, the role of ZIH in the project.

Large AI language models, such as GPT-3 from the American company OpenAI, which revolutionized the market about 2 years ago but does not offer its users free access but only an interface as access, or such as the Chinese model Wu Dao 2.0, use enormous amounts of data to create diverse AI models. These models, for example, write journalistic texts, support in customer service and help to understand complex texts. The problem is that data protection and the integration of European languages are not guaranteed.

* The American company OpenAI revolutionized the artificial intelligence market around two years ago with its GPT-3 language model. The model has been trained with over 175 billion parameters and thus beats previous models by a factor of ten. More than 300 applications already use the language model. However, the applications do not have free access to the model, but can only access an interface.

Contact at ZIH:
Prof. Dr. Wolfgang E. Nagel und Dr. René Jäkel