NewsImages Workshop 2023 in Amsterdam and Online

Author: Andreas Lommatzsch

The 14th Annual MediaEval Workshop has been held 1-2 February 2024. MediaEval has been held in conjunction with MMM in Amsterdam, Netherlands. MediaEval serves as a benchmarking initiative dedicated to fostering the creation and assessment of innovative algorithms within the multi-media field. It highlights the interdisciplinary nature of multimedia by offering datasets that combine different types of data, including text, audio, images, and video. In 2023, MediaEval introduced the following five tasks:

• Medical Multimedia Task – Transparent Tracking of Spermatozoa
• Musti: Multimodal Understanding of Smells in Texts and Images
• NewsImages: Connecting Text and Images
• Predicting Video Memorability
• SportsVideo: Fine Grained Action Classification and Position Detection in Table Tennis and Swimming Videos

Similar to last year’s edition, the DAI-Lab of the TU Berlin has been actively involved in the organization of the NewsImages lab.
The participants of NewsImages develop algorithms for re-establishing the connection between texts and images extracted from news portals. Based on a set of news articles (consisting of texts and an images), a set of texts and a set of images is defined. The lab participants have to reconstruct the connection between texts and images (originally defined by the news editors). As a special challenge, we partially replaced the original images by AI-generated image (using Stable Diffusion).

In 2023, 10 teams actively participated in NewsImages task. The teams submitted a up to 5 solutions for the news text -images re-matching task. Overall, the submitted solution very successfully solved the task. The highest performance has been reached by applying OpenCLIP. The winner team used extended fine tuning for optimizing the model to the concrete scenario.
An addition several BLIP and LLM-based methods have been suggested. The methods showed in average a lower performance than CLIP, but give new insights in the patterns used for finding good images for news articles.

The next edition of MediaEval is planned for October 2025 in Dublin. The following pictures give an impression of the MediaEval 2023 in Amsterdam.