NewsREEL @ CLEF 2015 in Toulouse

Author: Andreas Lommatzsch

The 2015 conference of the CLEF association was held in Toulouse (France). CLEF is an evaluation campaign stimulating the research on challenging questions in the Information Retrieval and related domains. CLEF focuses on evaluation methodologies and datasets. There are currently seven different labs each offering challenges and datasets.

CC IRML in cooperation with the CrowdRec project, the University of Glasgow and the plista GmbH organized the NewsREEL lab. The challenge in the NewsREEL lab consists in developing recommender algorithms for a real-time news recommendation scenario. The NewsREEL lab provides a 100GB data set of recorded user-news interactions (“offline task”) as well as an API enabling the live evaluation of recommender algorithm (“online task”). In 2015, 41 teams registered for the NewsREEL lab. In the NewsREEL lab workshop I presented the evaluation results of the participants and discussed the lessons learned while organizing the lab. Subsequently to two paper presentations, Roberto Turrin from contentwise gave an introduction to the Idomaar framework simplifying the offline evaluation of complex recommender components.

On Thursday, I presented the Paper “Optimizing and Evaluating Stream-based News Recommendation Algorithms” discussing approaches for developing new recommender algorithms optimized for multi-core machines.

The CLEF conference gave me the chance for countless fruitful discussion with researchers from all over the world. Unfortunately, there has not been much time left for visiting old famous churches of Toulouse or walking tours along the river Garonne. But there is one good message: NewsREEL will be CLEF lab in 2016. The CLEF 2016 conference will be organized in Evora (Portugal), 5.-8. September.