An open-access paper entitled “Pre-training Aware Directed Reasoning for Closed-Book Knowledge Base Construction” by Mikołaj Tym, a member of our Department, has been published. The paper introduces a method that uses large language models to automatically complete knowledge bases. The research was presented at a workshop held as part of the international EMNLP 2026 conference, which took place in Budapest, Hungary, from October 24 to 29, 2026.
The capabilities of large language models (LLMs) can be significantly enhanced by connecting them to external knowledge bases, for example through retrieval-augmented generation (RAG). However, maintaining such knowledge bases requires considerable effort. Therefore, an important area of research is exploring how to leverage knowledge encoded directly in the parameters of language models without relying on external information sources.
The research involved developing a method for automatically extracting facts from large language models. Its effectiveness was evaluated across six categories of facts, including the areas of geographical entities and the recipients of various awards. Another important aspect of the study was a comparison of 18 open-weight language models with up to 32 billion parameters.
The results showed that models performing well on widely used benchmarks (e.g., coding tasks) do not necessarily achieve the best results in automated knowledge base construction. Furthermore, the largest model evaluated was not the best-performing model for this task. The proposed method also achieved a score on the test set that was more than twice as high as that of the official baseline.
The paper was published in the proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026), dedicated to automated knowledge base construction. The workshop was held in conjunction with EMNLP 2026 (Conference on Empirical Methods in Natural Language Processing), one of the leading international scientific conferences in the field of natural language processing. The full paper is available through the ACL Anthology. DOI: doi.org/10.18653/2026.akbc-1.35