Constrained Retrieval-Augmented Language Models
Project Goals
The CORAL project aims to research methods for the construction and use of large language models (LLMs) that are subject to legal, technical, and qualitative constraints. With the fulfillment of legal requirements for the training data of LLMs and the referential provenance of the generated texts, our focus is on two central criteria that are indispensable for the professional use of language models. To this end, we are researching new methods for the constrained training of LLMs and the retrieval augmented generation (RAG) of texts.
Data
CORAL uses data from the partners involved. This includes the digital holdings of the German National Library (DNB), and web crawls from the Internet Archive and the Common Crawl, amounting to petabytes, many years of European-language news crawls from Wortschatz Leipzig, and proprietary data from the financial sector. Apart from the Common Crawl, this data has so far not been usable for training LLMs, as it is not made publicly available in its original form for legal reasons. We are therefore investigating the extent to which this data can be used legally for the training of LLMs in obfuscated form and how far the obfuscation of the data may go in order to construct useful large language models.
Research Questions
Central research questions are: Which training methods and model architectures are robust against data constraints? How resource-efficient can useful large language models be trained? Which methods of obfuscation, un-learning and negated augmentation effectively prevent the disclosure of protected data? How can the transparency and soundness, originality, and referenceability of the generated texts be ensured? How vulnerable are the methods used to secure the training data of LLMs?
CORAL is thus making important contributions to the future establishment of a German market for large language models.
Events
Keynote Presentation. Who writes the web? Who reads it? Who judges it? And who reports back to us? About authenticity in information retrieval. Martin Potthast at 15th International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR 2025) in July 2025
Poster Presentation. Learning Effective Representations for Retrieval Using Self-Distillation with Adaptive Relevance. Lukas Gienapp at 15th International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR 2025) in July 2025
Conference Talk. Axioms for Retrieval-Augmented Generation. Jan-Heinrich Reimer at 15th International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR 2025) in July 2025
Conference Talk. The Viability of Crowdsourcing for RAG Evaluation. Lukas Gienapp at 48th International ACM SIGIR Conference on Research and Development in Information Retrieval in July 2025
Invited talk. Retrieval Technologies for the Infinite Index. Martin Potthast at the 21st Conference on Database Systems for Business, Technology and Web (BTW) in Bamberg in March 2025.
Invited talk. Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models. Christopher Schröder at the MLT Meetings at DFKI (Online) in March 2025.
Publications
Awards
Best Paper Honourable Mention Award. For "The Viability of Crowdsourcing for RAG Evaluation". At 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2025)
Best Paper Honourable Mention Award. For "Axioms for Retrieval-Augmented Generation". At 15th International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR 2025)
