TrustLLM - Large Language Model Technology for Europe

Natural Language Processing (NLP) is increasingly important in our digital lives. In particular, Large language models (LLMs) such as GPT-4 and chatbots like ChatGPT are likely to profoundly impact society, affecting an estimated 80% of the workforce. However, the current technology is still immature in several essential respects.

The recently funded Horizon Europe Project TrustLLM aims to conduct the research and development necessary to align LLMs with the European vision of human-centred, trustworthy, and democratized AI. This Regular Access project provides the TrustLLM EU project and its LLM experts with the required access to computing resources. High-Performance Computing (HPC) plays a crucial role in training different variants of LLMs relevant to the TrustLLM project, each of which needs substantial GPU resources to train large-scale deep neural networks on trillions of tokens.

The highly experimental nature of this project will likely give rise to many model architectures that need to be explored. The major expected outcomes are: (i) knowledge of which model architectures are most suitable for LLM foundation models that are openly shared with the EU LLM community; (ii) research and scaling experiments for increasing model sizes (e.g., better model capabilities) and decreasing model sizes (e.g., better energy efficiency).