Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE
CLAIMSCAN 2023: UniBw and Fraunhofer FKIE methods for identifying and evaluating claims take first place
More success for the Fraunhofer FKIE scientists in Wachtberg: The NLytics system was declared the winner of the newest shared-task competition CLAIMSCAN 2023, organized by the Indian Institute of Technology Delhi. The system is the result of a joint research project in collaboration with the University of the Bundeswehr Munich and aims to detect propaganda and misinformation campaigns on social media using AI.
“The success of social media has entirely changed the communication and media landscape. This has led to a surge in all kinds of content being produced and shared,” explains FKIE project manager and CLAIMSCAN participant Albert Pritzkau. This is not entirely a positive development as these platforms also provide opportunities to spread rumors, fake news, propaganda and misinformation. The scientist says that the sheer quantity of this information alone makes it impossible to identify these false claims effectively. Automatically differentiating between true and false claims is therefore an important research objective in computational linguistics.
At the CLAIMSCAN shared-task competition, organized by the Indian Institute of Technology Delhi for the first time in 2023, researchers from across the world had the opportunity to test the mettle of their methods. The winner was declared as the German NLytics system, developed through a joint research project involving the University of the Bundeswehr Munich and Fraunhofer FKIE.
“The project focused on identifying targeted, manipulative fake news by using AI,” explains Pritzkau. In addition to the project, the scientist also worked intensively on the issue as part of his doctoral studies. With NLytics, he developed a new approach that expands upon a modern machine learning system to improve how linguistic characteristics are taken into consideration. Pritzkau also used explainable AI (XAI) to clean the training data. According to the researcher, “AI systems are notoriously only as good as their training data.” Pre-processing the training data according to his approach significantly improves the results and quality of the system.
NLytics secured the team the top spot in one of the two CLAIMSCAN shared tasks. The competition success further validates the results of the scientist’s doctoral studies: Last September, Pritzkau’s concept was awarded best in category in the DIPROMAT 2023 shared-task competition run by the Iberian Languages Forum (IberLEF), which is aimed at identifying propaganda.