NewsREEL Multimedia @ MediaEval Benchmark 2018

Author: Andreas Lommatzsch

The MediaEval Benchmarking is an Initiative for Multimedia Evaluation offering data and challenges in the form of shared tasks. MediaEval focusses on innovative algorithms and methods for exploring and retrieval multi-media data. The provided data typically cover several modalities such as audio-visual (video, audio), textual, and contextual data.

NewsREEL Multimedia (News Recommendation with Image/Text Contents) runs as a new pilot task in 2018. The task is based on the observation that images have a big impact on perception of news. The NewsREEL MultiMedia provides a data collection of news articles, item popularity data, and (compressed) image data. The task consists in predicting the popularity of new items based on the multi-media data. The quality of the predictions is measured with the Precision and the NDCG.

The MediaEval workshop 2018 will be held at EURECOM in Sophia Antipolis, France, in November 2018.

Web page: MediaEval-Benchmark

Web page: MediaEval-Benchmark

Tutorial Real-time Recommendations with Streamed Data @ KI2018

Author: Andreas Lommatzsch

The 41st German Conference on Artificial Intelligence will be held September 24-28th, 2018, Berlin, Germany. The conference brings together academic and industrial researchers from all areas of AI. It is an well-established venue to discuss new trends in AI and to exchange new research results. The deadline for submitting abstracts is on April 30th, 2018.

Tutorials and workshops are scheduled for the first two days of the conference. We plan to give a tutorial on Real-time Recommendations with Streamed Data. The tutorial introduces a combined view on multi-dimensional benchmarking and real-time requirements in recommender systems. Participants will not only learn about recommendation algorithms but also delve into practical issues related to system maintenance, model updating, and continuous evaluation. For more details please visit the tutorial webpage.



This week in AI #42

With this post, we want to start sharing some of the interesting news regarding the topic of AI. Regarding the fact that we are a research group based in Germany, we will also post articles and references in German – feel free to use for translation…

And starting with some news in German and from Germany:

  • Wenn der Roboter Wünsche von den Augen abliest

    Die International Conference on Human-Agent Interaction (HAI 2017) findet in Bielefeld statt und beschäftigt sich mit der Kommunikation zwischen Mensch und Maschine und der Erkenntnis, dass Kommunikation nicht nur auf Worte beschränkt ist.
  • Emirate richten Ministerium für Künstliche Intelligenz ein

    “Die nächste globale Welle ist die künstliche Intelligenz. Wir wollen das Land sein, das darauf am besten vorbereitet ist”, schrieb Ministerpräsident Mohammed bin Raschid Al Maktum am Donnerstag auf Twitter. Dazu nur der Hinweis auf den Zeit Artikel vom 28.09.2017 – Diese Überschrift muss leider noch laden ….
  • A brief guide to mobile AI chips

    A reminder that the current AI (re)evolution is not only about the algorithms but also about the hardware. And while the author is making some good points for specialized hardware, the conclusion “SO DO I NEED AN AI CHIP IN MY PHONE? – No, not really. So much work is being done on making AI services run better on the hardware currently available, that unless you’re a real power user, you don’t need to worry about it.” is the wrong one. We instead need more efforts to bring specialized hardware (and software) to products to improve the usefulness of AI services. Everybody who has tried using Siri or any other smart assistant without a stable internet connection will probably agree. The current detour over the cloud is because we lack proper hardware on the mobile devices.
  • The BBC is turning to AI to improve its programming

    Kudos to the BBC for constantly exploring new paths. I remember that BBC was also the first TV station I heard of trying out Semantic Web standards. One idea BBC is researching as “a concept called object-based broadcasting…that could be assembled in different ways depending on the user or end-hardware” – the dream of truly personalized TV broadcasting.

Till Plumbaum

8th Conference of the CLEF Initiative in Dublin

Author: Andreas Lommatzsch

The 8th conference of the CLEF initiative has been held at the Trinity College in Dublin from September 11th till September 14th. The CLEF initiative is a European campaign focusing on evaluation methods in Information Retrieval and related fields.

The News Recommendation Lab (NewsREEL) runs as a CLEF lab the third year. NewsREEL gives researchers the ability to evaluate news recommendation algorithm both online and offline. The stream-based data, the use multi-dimensional benchmarking (taking into account precision and technical metrics) as well as the evaluation based on live user feedback make NewsREEL special CLEF lab.

In the NewsREEL workshop the participating teams presented the evaluated algorithms. The spectrum of implemented solutions covered a wide spectrum of approaches ranging from framework-based approaches (using abstract query languages) to more experimental approaches (using Contextual Bandits). The enabled a vivid, fruitful discussion between the participating teams. Detailed descriptions of the approaches are explained in the working notes.

Co-located with CLEF 2017, the yearly conference of the MediaEval Benchmarking Initiative was held. Several joint conference sessions and poster sessions provided the basis for interesting discussions and the exchange of ideas. Hopefully, the cooperation between the two benchmarking initiatives will continue in the future.

NewsREEL will continue also in 2018. It will not run as a CLEF lab in order to focus stronger on data mining and recommender algorithms.

The following photos give a visual impression from the Trinity College Campus in Dublin as well as from the CLEF conference and MediaEval conference.

ACM Conference on Recommender Systems 2017

The 11th edition of the ACM Conference on Recommender Systems will take place in Como, Italy, in the time 27-31 August, 2017. The conference features three keynotes, a panel discussion, eleven paper and three industry sessions, three tutorials, and thirteen workshops. On Tuesday, 29 August 11 a.m., Benjamin Kille will present the article “Recommending Personalised News in Short User Sessions” co-authored with Elena Epure, Rebecca Deneckere, Camille Salinesi (Panthéon-Sorbonne), Jon Espen Ingvaldsen (formerly NTNU), and Sahin Albayrak (DAI-Labor). The article analyses how category information can support news recommender systems to select a more relevant set of suggested readings. On of the workshops, the Temporal Reasoning in Recommender Systems Workshop, features another CC-IRML publication. The article “On the Decaying Utility of News Recommendation Models” by Benjamin Kille and Sahin Albayrak shows the necessity to update recommendation models in the domain of news. Its presentation is scheduled on Thursday, 31 August, 9:50 a.m.

International Conference on Web Intelligence

The IEEE/WIC/ACM International Conference on Web Intelligence will take place 23-26 August, 2017, in Leipzig, Germany. The conference is co-located with the 15th German International Conference on Multiagent Systems Technologies (MATES). It features six keynotes, seven paper sessions, seven tutorials, and nine workshops one of which is the International Workshop on News Recommendation and Analytics. Among its publications, the article “Incorporating Context and Trends In News Recommender Systems” by Andreas Lommatzsch and Benjamin Kille explores how contextual factors and trends can help readers to find more relevant news articles. The talk is scheduled on Wednesday, 23 August, 2017 at 10 a.m.

I4CS conference 2017, Darmstadt, Germany

Author: Andreas Lommatzsch

The 17th edition of the International Conference on Innovative Internet Community Systems (I4CS) was held at T-Systems in Darmstadt 26-28 June 2017. The conference focuses on Internet Community Systems including distributed architectures, Social Networks and open collaboration services as well as on eHealth and intelligent transport solutions.

I presented datasets and methods for the sentiment analysis of German News and Forum texts. My talk discussed the challenges of automatically determining the sentiments of German texts with respect to the specific application scenario. The developed approach and the implemented systems have been discussed in detail.

Many interesting papers were presented at the I4CS conference. I would like to mention the talk "On a Fog Computing Platform Built on ARM Architectures by Docker Container Technology" given by Udo Krieger. The paper discusses the concepts of a fog computing platform extending the principles and services of cloud computing to the edge of a network. The talk gave a good overview on the platform design principles and the implemented software components optimized for ensuring highly-available, efficiently working fog cells. The talk was awarded with the best presentation award.

Additional highlights of the conference were a visit of the QIVICON House, a tour over the Technologie-Zentrum Rhein Main business park, and the sightseeing tour through the city of Darmstadt.

The 18th I4CS conference will be held in Zilina, Slovakia.

Big Data Universe 2017 in Budapest

The Big Data Universe 17 in Budapest took place in the Aqvarium Club, an exclusive location in the very middle of the city, located below a glass roof right under a public pool. The conference gathered top influencers of the hungarian big data industry, interested in state of the art products and research directions. A wide range of speakers like Lucas Bernardi (, Dániel Molnár (Door2Door GmbH) Attila Komor (Nextent Informatics Co.) and Mátyás Dobó (Telekom) provided insights into the challenges and solutions of their daily work.

Besides the conference talks, we where able to present our newest meta machine learning prototype, which is a part of the projects under the roof of the new Berlin School of AI, a cooperation between the DAI-Labor and T-Labs . It is the first step towards gathering reproducable performance data of machine learning algorithms in order to compare not just the predictive performance, but also the costs it takes to use them. It enables decisions about efficiency, not just effectivity. In the future we want to go beyond and use the gathered data for model selection and tuning approaches based on meta features.

We where happy to show our work and to get valuable feedback from experts from all over the place. Very special thanks go out to Dr. Tomáš Horváth and his team, who where a great help and showed us around.


Something for the weekend

Google presented a great demo to play a piano duet with a computer. So if you have time on the weekend and no company, maybe Tensorflow is a good match.

Check out Magenta

NewsREEL 2017

Author: Benjamin Kille

We are excited to announce the start of another edition of CLEF NewsREEL. This year’s challenge continues with both tasks:

NewsREEL Live: interact with an operating news recommender systems and real users
NewsREEL Replay: analyze a comprehensive data set to determine the best recommender

Our partners at plista are currently busy revising the Open Recommendation Platform (ORP). The new systems will be more stable and equipped with a new UI for a much better user experience. Until the new systems has been released, we encourage participants to engage in NewsREEL Replay.

We released a new data set of more than 168 million events recorded over the stretch of four weeks. The data set provides a representative picture of what your recommender will encounter in NewsREEL Live. The following charts gives you an impression on what the data looks like.




If you want to join us in the pursuit of news recommender systems evaluation, please register at here and make sure to check the boxes for NewsREEL. You will receive information about how to access the data shortly after successful registration.