NewsImages at MediaEval 2022 in Bergen, Norway

Author: Andreas Lommatzsch

The MediaEval conference 2022 has been held in Bergen, Norway. The conference has been co-located with the 29th International Conference on Multimedia Modeling to foster an exchange of ideas between the communities. The NewsImages is part of MediaEval since 2018. NewsImages researches methods for analyzing the use of multimedia in the news domain.

This year, NewsImages provided three news datasets from three different domains: (1) The RSS part provided news from different news portals (aggregated by the dgelt-Project). The second part TW provided news stories-related tweets. The third part RT provided news in German related to the war in Ukraine. The task in NewsImages consists in re-matching news text and images. Details about the analyzed setting, the dataset, and the evaluation metrics are explained in the Lab overview paper.

In this year’s challenge, 5 teams from Europe and Asia participated. The developed solutions provided very good results, The best team reached a Recall@5 =0.6 . The discussion of the results gave interesting insights in the relation between the news texts and the used images. The best results were obtained by annotating images using CLIP. For correlating text and image considering only a short text (e.g. the headline) has been most successful. This underlines that image and headline are often based on the same key element of the news message. The best results have been reported for tweets indicating that there is a strong overlap between the (relatively) short text and the accompanying image. The typically required translation step for the texts from the RT dataset reduced the reached recall by about 15%. The teams in the challenge mainly focused on optimizing the similarity models for connecting texts and images as well as on methods for bridging the semantic gap between news texts and images by enriching detected concepts with semantic information.

Details about the implemented methods can be found in the MediaEval proceedings.