In 2024, scientists from the Department of Information Systems participated for the second time in the CheckThat! competition, part of the international CLEF conference, focused on identifying check-worthy texts. Leveraging transformer-based language models and cross-linguistic transfer learning techniques, the team secured two second-place rankings for English and Arabic submissions.
It’s noteworthy that this highly competitive event attracted 26 teams from around the globe. The method developed by the scientists of our Department was outperformed only by teams from Queen Mary University of London (for English) and the University of Stavanger (for Arabic). Participants included prestigious institutions such as the Fraunhofer Institute (Germany), Rijksuniversiteit Groningen (Netherlands), Chittagong University of Engineering and Technology (Bangladesh), University of Jaén (Spain), Polish-Japanese Academy of Information Technology (Poland), Qatar University (Qatar), Sorbonne Université (France), Airbus Defence and Space (France), and University of Montréal (Canada). Detailed information about this year’s edition of the competition is available in the publication titled “Overview of the CLEF-2024 CheckThat! Lab Task 1 on Check-Worthiness Estimation of Multigenre Content”.
Check-worthy texts contain statements that can be verified against external, trusted sources and may be potentially harmful. Identifying such texts is a crucial aspect of the fact-checking process, as it enables the selection of messages from vast collections of social media and online portal information, which can then be verified by professional fact-checking agencies.The competition evaluated F1 scores for the positive class in English, Arabic, and Dutch corpora. Leveraging transformer-based language models and cross-linguistic transfer learning techniques, the team secured two second-place rankings for English and Arabic submissions. Further details can be found in the publication by Marcin Sawiński, Prof. Krzysztof Węcel and Ewelina Księżniak, titled “OpenFact at CheckThat! 2024: Cross-Lingual Transfer Learning for Check-Worthiness Detection“.
This marks the third publication by our scientists on detecting information worth checking, as part of the CLEF conference. Last year, the team earned first place using fine-tuning techniques on large language models. Additionally, their 2023 publication was honored with the Best of Labs title, leading to an invitation for the authors to further develop their research and present in the main section of the CLEF 2024 conference.
The “CLEF CheckThat! Lab” event, held since 2018, aims to develop automated methods and technologies to support journalists in verifying information. The international CLEF conference has been hosted in cities such as Bologna, Bucharest, Lugano, Avignon, and Thessaloniki. This year’s results will be presented in September at a conference in Grenoble, France. Alongside “FEVER,” “CLEF CheckThat! Lab” is one of the most significant global events dedicated exclusively to the automatic verification of fake news.
It is worth mentioning that this year, scientists from our Department achieved a high position not only in the competition concerning the detection of check-worthy texts, but also took first place in competitions in the area of information credibility and achieved the best result in the area of early risk prediction on the Internet. Department of Information Systems is currently implementing the OpenFact research project, within which it is developing tools for automatic detection of fake news in Polish. In July 2024, the results of the OpenFact project were rated as the best in Poland by National Center for Research and Development for the second year in a row.
The OpenFact project is financed by the National Center for Research and Development under the INFOSTRATEG I program “Advanced information, telecommunications and mechatronic technologies”.