Author: Benjamin Kille
We are excited to announce the start of another edition of CLEF NewsREEL. This year’s challenge continues with both tasks:
NewsREEL Live: interact with an operating news recommender systems and real users
NewsREEL Replay: analyze a comprehensive data set to determine the best recommender
Our partners at plista are currently busy revising the Open Recommendation Platform (ORP). The new systems will be more stable and equipped with a new UI for a much better user experience. Until the new systems has been released, we encourage participants to engage in NewsREEL Replay.
We released a new data set of more than 168 million events recorded over the stretch of four weeks. The data set provides a representative picture of what your recommender will encounter in NewsREEL Live. The following charts gives you an impression on what the data looks like.
If you want to join us in the pursuit of news recommender systems evaluation, please register at here and make sure to check the boxes for NewsREEL. You will receive information about how to access the data shortly after successful registration.
Author: Andreas Lommatzsch
The Recommender Meetup Amsterdam (www.recommenders.nl) is meetup focusing on innovative recommender algorithms and AI methods. The meetup is organized by the TU Delft and hosted by different companies in the Netherlands. The 6th RecSys:NL Meetup was held on September 28th at ORTEC Amsterdam.
There were three talks: First, Alexandros Karatzoglou (Telefonica Research, Barcelona, Spain) gave a talk about Deep Learning Methods and the use of Deep Leaning approaches in Recommender Systems. In his talk, he discussed the state of the art of Deep Learning methods and explained current trends and application scenarios in the Deep Learning methods are used.
The second talk was given by Andreas Lommatzsch (TU Berlin, Berlin, Germany). In his talk "NewsREEL challenge: Online evaluation of recommender systems" he explained the CLEF NewsREEL challenge and the challenges of online evaluation methods. The CLEF NewsREEL challenge allows researchers to evaluate algorithms for recommending online news articles. The talk explained the NewsREEL scenario in detail. The specific challenges have been discussed such as the continuous changes in the set of relevant items, the influence of context parameters, and the variance in the user behavior. In addition, the talk presented different algorithmic approaches as well as the experiences gained organizing the challenge. The slides are available here.
Daniel Kohlsdorff (XING, Hamburg, Germany) explained the challenges of providing high-quality job recommendations. He presented selected components of the XING’s recommender system explaining content based recommendations with latent spaces, and outlier filtering. In addition, Daniel presented the results of the RecSys challenge 2016 (co-organized by XING) and gave an outlook on the RecSys challenge 2017.
After the talks there has been a lively discussion accompanied by pizza and beer.
Online Evaluation of Recommender Systems
Amsterdam at night
Welcome to the Recommender Meetup Amsterdam
Deep learning in recommender systems
Recommendation challenges on XING
Author: Andreas Lommatzsch
The LWDA ("Lernen Wissen Daten Analysen") conference is a conference organized by the Gesellschaft für Informatik (German Informatics Society). The 2016 edition was held in Potsdam, 12-14 September 2016.
The conference is organized in 4 tracks focusing on Information Retrieval (IR), Knowledge Discovery and Machine Learning (KDML), Databases (DB) and Knowledge Management (WM).
CC IRML contributes to the conference in three different tracks.
The paper "An Interactive e-Government Question Answering System" presents a Question-Answering system developed for the specific requirements of an e-government scenario. The system combines information retrieval and AI methods for providing direct answers to questions concerning governmental services. The systems has been developed in the master course "Semantic Search Project".
The paper "Topical Video-On-Demand Recommendations based on Event Detection" presents an event-based recommender system for art house movies Based on semantic sources and social media events are identified that are used for computing movie recommendations related to specific days and events.
The poster "Applying Topic Model in Context-Aware TV Programs Recommendation" presents our approach for learning context aware recommendations combining topic models and model built based on a user data-based Latent-Dirichlet Allocation.
The very active poster session and the high-quality keynote talks as well as the modern venue (HPI, Campus Griebnitzsee) inspired valuable discussion and new research ideas.
The HPI, campus Potsdam Griebnitzsee
Our poster explaining the event-based recommender apporach
The poster for the knowledge management track.
The first slide of the Q&A paper presentation.
Our poster "Applying Topic Model in Context-Aware TV Programs Recommendation"
Author: Till Plumbaum
At this years RecSys we presented a demo system that collects topics of interest from Twitter and RSS-Feeds, extracts relevant Named Entities, and uses semantic relations for recommending movies closely related to these topics.
During the poster session, which was running for 3 1/2 hours, we were constantly in discussion with the great RecSys crowd.
You can get the poster here. If you are interested in the topic, we are happy to get in touch with you, just send a mail to the authors and we will get back to you.
The research leading to these results was performed in the CrowdRec project, which has received funding from the EU 7th Framework Programme FP7/2007-2013 under grant agreement No. 610594.
You can find all the answers on: http://gamifir.com/
Author: Till Plumbaum
We presented our CLEF Newsreel Challenge at the Algorithms with Friends II Meetup in Berlin with around 100 participants. Andreas talked about the challenges and findings of the third year of NewsReel, which focused on offline vs online evaluation and stream recommendations. We got very positive feedback from the crowd and are now encouraged to actively plan the 4th incarnation of the challenge.
You can find the slides here – NewsReel_Meetup.pdf
And some impressions from the meetup:
Author: Till Plumbaum
In March I presented the Deutsche Bahn Digital Header Challenge at the Algorithms & Data Challenges meetup.
On July 11th, the next Algorithms & Data Challenges meetup is hosted by Zalando and we are returning to present results. I will present results from the DB Challenge and Andreas Lommatzsch will present results from the CLEF challenge that we are also organizing with focus on news recommendation in a streaming scenario.
You can find more information about the meetup here
And a few impressions from the last meetup:
Third International Workshop on Gamification for Information Retrieval (GamifIR 2016)
Held in conjunction with SIGIR 2016
21 July 2016
Call for Papers – Deadline: May 29, 2016
Many research challenges in the field of IR rely on tedious manual labour. Canonical examples include the manual feedback required to assess the relevance of documents to a given search task or the evaluation of interactive IR approaches. Performing these tasks through Crowdsourcing techniques can be useful, but often fail when motivated users are required to perform a task for reasons other than just being paid per click. A promising approach to increase user motivation is by employing Gamification methods which has been applied in various environments and for different purposes, such as marketing, education, pervasive health care, enterprise workplaces, e-commerce, human resource management and many more.
The Third International Workshop on Gamification for Information Retrieval (GamifIR 2016) focuses on the challenges and opportunities that Gamification can present for the IR community. The workshop, organized in conjunction with SIGIR 2016, aims to bring together researchers and practitioners from a wide range of areas including game design, information retrieval, human-computer interaction, computer games, and natural language processing.
Topics of Interest
In this workshop we want to discuss and learn from planned and already executed studies and utilization of Gamification. Therefore we invite the submission of position papers as well as novel research papers and demos addressing problems related to Gamification in IR. Topics of interest include but are not limited to:
- Gamification approaches in a variety of contexts, including document annotation and ground-truth generation; interface design; information seeking; user modelling; knowledge sharing
- Gamification in Crowdsourcing
- Gamification to IR teaching
- Gamification in recommender systems and apps
- User engagement and motivational factors of Gamification
- Player types, contests, cooperative Gamification
- Long-term engagement
- Search challenges
- Gamification design
- Applied game principles, elements and mechanics
Submissions from outside the core IR community and from industry are actively encouraged.
All papers should be 2-6 pages long (in PDF format) following the ACM proceedings format. LaTeX and Word templates are available at: http://www.acm.org/publications/proceedings-template. Submissions will be peer-reviewed by at least three members of the program committee. We plan to publish the papers as CEUR Workshop Proceedings.
Please submit your papers in PDF format through: https://easychair.org/conferences/?conf=gamifir2016
We aim for an interactive and dynamic full-day workshop that will be a mix of keynotes, paper and demo presentations as well as discussion sessions.
- Omar Alonso, Microsoft Research (USA)
- Raian Ali, Bournemouth University (UK)
- Leif Azzopardi, University of Glasgow (UK)
- Jon Chamberlain, University of Essex (UK)
- Sebastian Deterding, Northeastern University (USA)
- Carsten Eickhoff, ETH Zurich (CH)
- Christopher G Harris, State University of New York (USA)
- Hideo Joho, University of Tsukuba (JP)
- Till Plumbaum, TU Berlin (GER)
- Ashok Ranchod, University of Southampton (UK)
- Thomas Springer, TU Dresden (GER)
- Susanne Strahringer, TU Dresden (GER)
- Albert Weichselbraun, University of Applied Sciences Chur (CH)
- Lincoln Wood, The University of Auckland (NZ)
- Frank Hopfgartner, University of Glasgow (United Kingdom)
- Gabriella Kazai, Semion Ltd. (United Kingdom)
- Udo Kruschwitz, University of Essex (United Kingdom)
- Michael Meder, TU Berlin (Germany)
- Submission deadline: May 29, 2016
- Notification date: June 19, 2016
- Camera-ready due: July 3, 2016
- Workshop date: July 21, 2016
Author: Andreas Lommatzsch
The European Conference on Information Retrieval 2016 was held in Padua (Italy) from March 20th to March 23rd, 2016. The topics of the conference cover a wide spectrum ranging from classical IR algorithms evaluated on static dataset, to Question Answering, to Evaluation Methodologies and Machine Learning.
The opening keynote was given by Jordan Boyd-Graber (University of Colorado, USA). The interesting talk "Opening up the black box: Interactive Machine Learning" discussed the role of components built based on AI algorithm in the society. In order to trust in these systems, transparent machine learning algorithms are needed, especially for complex tasks. Improving the transparency and the provided explanation is more important that improving the precision for providing the optimal support for decisions made by humans.
On Tuesday Emine Yilmaz (University College London, UK) gave the keynote titled "A Task-based Perspective to IR". The talk outlined that IR should focus on task trees. In contrast to simple list of documents ranked by assumed relevance, task trees are powerful tools supporting users in solving complex problems. The talk summarized the state-of-the art and outlined current trends in this domain.
On the third day of the conference, I attended the Industry session. Domonkos Tikk (Gravity R&D, Hungary) gave the keynote "Lessons learned at Building Recommender Services in Industry scale". The very interesting talk explained the challenges of adapting algorithms for real-life applications and strategies for handling the technical complexity of algorithms.
In the subsequent session Jonas Seiler (plista), Daniel Kohlsdorf (XING) and I presented the talk "Get on with it – Recommender system industry challenges move towards real-world, online evaluation". In the presentation we suggested approaches for applying more realistic evaluation settings. Starting with the evolution of the evaluation approaches in IR and recommender systems we outlined the challenges evaluating real-life systems taking into account not only the precision of the results but also technical aspects and business models. We suggested a closer cooperation between academia and industry to their mutual benefit e.g., by participating in challenges organised by industry and academia. The evaluation should consider heterogeneous data as well as, streamed data, and focus on multi-dimensional metrics taking into account the needs of users and the service providers. The RecSys Challenge 2016 as well as the CLEF NewsREEL-Challenge offer interesting opportunities for evaluating own algorithms in real-life scenarios.
The venues of the conference have been impressive: At the first day, the conference talks took place in the Botanical Garden. The second and third were held in the historical university building next to the town hall. In the "Aula Magna" Galileo Galilei gave lectures in the 16th century.
The following photos give a visual impression of the ECIR 2016.
Botanical Garden Padua
ECIR 2016 – opening
Poster session @Cafe Pedrocchi
University Padua – Aula Magna
Keynote talk @ ECIR 2016
Padua – Prato della Valle