6th Recommender Meetup Amsterdam

Author: Andreas Lommatzsch

The Recommender Meetup Amsterdam (www.recommenders.nl) is meetup focusing on innovative recommender algorithms and AI methods. The meetup is organized by the TU Delft and hosted by different companies in the Netherlands. The 6th RecSys:NL Meetup was held on September 28th at ORTEC Amsterdam.

There were three talks: First, Alexandros Karatzoglou (Telefonica Research, Barcelona, Spain) gave a talk about Deep Learning Methods and the use of Deep Leaning approaches in Recommender Systems. In his talk, he discussed the state of the art of Deep Learning methods and explained current trends and application scenarios in the Deep Learning methods are used.

The second talk was given by Andreas Lommatzsch (TU Berlin, Berlin, Germany). In his talk "NewsREEL challenge: Online evaluation of recommender systems" he explained the CLEF NewsREEL challenge and the challenges of online evaluation methods. The CLEF NewsREEL challenge allows researchers to evaluate algorithms for recommending online news articles. The talk explained the NewsREEL scenario in detail. The specific challenges have been discussed such as the continuous changes in the set of relevant items, the influence of context parameters, and the variance in the user behavior. In addition, the talk presented different algorithmic approaches as well as the experiences gained organizing the challenge. The slides are available here.

Daniel Kohlsdorff (XING, Hamburg, Germany) explained the challenges of providing high-quality job recommendations. He presented selected components of the XING’s recommender system explaining content based recommendations with latent spaces, and outlier filtering. In addition, Daniel presented the results of the RecSys challenge 2016 (co-organized by XING) and gave an outlook on the RecSys challenge 2017.

After the talks there has been a lively discussion accompanied by pizza and beer.

LWDA conference 2016, Potsdam, Germany

Author: Andreas Lommatzsch

The LWDA ("Lernen Wissen Daten Analysen") conference is a conference organized by the Gesellschaft für Informatik (German Informatics Society). The 2016 edition was held in Potsdam, 12-14 September 2016.
The conference is organized in 4 tracks focusing on Information Retrieval (IR), Knowledge Discovery and Machine Learning (KDML), Databases (DB) and Knowledge Management (WM).
CC IRML contributes to the conference in three different tracks.

The paper "An Interactive e-Government Question Answering System" presents a Question-Answering system developed for the specific requirements of an e-government scenario. The system combines information retrieval and AI methods for providing direct answers to questions concerning governmental services. The systems has been developed in the master course "Semantic Search Project".

The paper "Topical Video-On-Demand Recommendations based on Event Detection" presents an event-based recommender system for art house movies Based on semantic sources and social media events are identified that are used for computing movie recommendations related to specific days and events.

The poster "Applying Topic Model in Context-Aware TV Programs Recommendation" presents our approach for learning context aware recommendations combining topic models and model built based on a user data-based Latent-Dirichlet Allocation.

The very active poster session and the high-quality keynote talks as well as the modern venue (HPI, Campus Griebnitzsee) inspired valuable discussion and new research ideas.

CrowdRec: Topical Semantic Recommendations for Auteur Films @RecSys 2016

Author: Till Plumbaum

At this years RecSys we presented a demo system that collects topics of interest from Twitter and RSS-Feeds, extracts relevant Named Entities, and uses semantic relations for recommending movies closely related to these topics.
During the poster session, which was running for 3 1/2 hours, we were constantly in discussion with the great RecSys crowd.

You can get the poster here. If you are interested in the topic, we are happy to get in touch with you, just send a mail to the authors and we will get back to you.

The research leading to these results was performed in the CrowdRec project, which has received funding from the EU 7th Framework Programme FP7/2007-2013 under grant agreement No. 610594.

NewsREEL 2016 in Evora

Author: Andreas Lommatzsch

The NewsREEL challenge allows researchers to test recommender algorithms in a real-world scenario. The NewsREEL challenge provides an online task (Living Lab) in that news articles recommendations must be provided in real time. The recommendation quality is evaluated based on real user feedback using the Click-Through-Rate as performance metric. In the offline task the task organizers provide components for re-playing datasets collected in the online task. This allows researchers to investigate recommender algorithms in detail in an exactly reproducible setting. Since the online and the offline task use the same data format, researchers can analyze implemented algorithms both online and offline as well as investigate the correlation of the measured results. The main research focus lies on the multi-dimensional benchmarking of recommender algorithms based on dynamic, stream-based data. A detailed summary of NewsREEL lab can be found in the CLEF 2016 proceedings.

The NewsREEL runs 2016 as a lab of the CLEF conferences (Conference and Labs of the Evaluation Forum). Beside the NewsREEL challenge, there are several other challenges covering a broad spectrum of topics ranging from advanced text analysis in PAN (uncovering Plagiarism, Authorship and Social Software Misuse) to multi-media analysis in LifeCLEF to the social media analysis in the Cultural Microblog workshop. There has been a lot a very fruitful discussion giving ideas for new research approaches. CLEF 2017 will be held in Dublin (Ireland), 11-14 September 2017. NewsREEL will also run in 2017 as a CLEF lab.

The CLEF 2016 conference has been organized by the University of Évora, being the second oldest university in Portugal. Évora is a UNESCO World Heritage Site, known for the well-preserved old town center and a large number of historical monuments such as the Roman Temple. The following photos give a visual impression of the town and the conference.

GamifIR 2016 – What happened in Pisa?

You can find all the answers on: http://gamifir.com/

NewsReel at the Algorithms & Data Challenges Meetup

Author: Till Plumbaum

We presented our CLEF Newsreel Challenge at the Algorithms with Friends II Meetup in Berlin with around 100 participants. Andreas talked about the challenges and findings of the third year of NewsReel, which focused on offline vs online evaluation and stream recommendations. We got very positive feedback from the crowd and are now encouraged to actively plan the 4th incarnation of the challenge.

You can find the slides here – NewsReel_Meetup.pdf

And some impressions from the meetup:

Algorithms & Data Challenges Meetup

Author: Till Plumbaum

In March I presented the Deutsche Bahn Digital Header Challenge at the Algorithms & Data Challenges meetup.

On July 11th, the next Algorithms & Data Challenges meetup is hosted by Zalando and we are returning to present results. I will present results from the DB Challenge and Andreas Lommatzsch will present results from the CLEF challenge that we are also organizing with focus on news recommendation in a streaming scenario.

You can find more information about the meetup here

And a few impressions from the last meetup:




Third International Workshop on Gamification for Information Retrieval (GamifIR 2016)

Held in conjunction with SIGIR 2016
Pisa, Italy
21 July 2016

Call for Papers – Deadline: May 29, 2016


Many research challenges in the field of IR rely on tedious manual labour. Canonical examples include the manual feedback required to assess the relevance of documents to a given search task or the evaluation of interactive IR approaches. Performing these tasks through Crowdsourcing techniques can be useful, but often fail when motivated users are required to perform a task for reasons other than just being paid per click. A promising approach to increase user motivation is by employing Gamification methods which has been applied in various environments and for different purposes, such as marketing, education, pervasive health care, enterprise workplaces, e-commerce, human resource management and many more.

The Third International Workshop on Gamification for Information Retrieval (GamifIR 2016) focuses on the challenges and opportunities that Gamification can present for the IR community. The workshop, organized in conjunction with SIGIR 2016, aims to bring together researchers and practitioners from a wide range of areas including game design, information retrieval, human-computer interaction, computer games, and natural language processing.

Topics of Interest

In this workshop we want to discuss and learn from planned and already executed studies and utilization of Gamification. Therefore we invite the submission of position papers as well as novel research papers and demos addressing problems related to Gamification in IR. Topics of interest include but are not limited to:

  • Gamification approaches in a variety of contexts, including document annotation and ground-truth generation; interface design; information seeking; user modelling; knowledge sharing
  • Gamification in Crowdsourcing
  • Gamification to IR teaching
  • Gamification in recommender systems and apps
  • User engagement and motivational factors of Gamification
  • Player types, contests, cooperative Gamification
  • Long-term engagement
  • Search challenges
  • Gamification design
  • Applied game principles, elements and mechanics

Submissions from outside the core IR community and from industry are actively encouraged.

Submission Details

All papers should be 2-6 pages long (in PDF format) following the ACM proceedings format. LaTeX and Word templates are available at: http://www.acm.org/publications/proceedings-template. Submissions will be peer-reviewed by at least three members of the program committee. We plan to publish the papers as CEUR Workshop Proceedings.

Please submit your papers in PDF format through: https://easychair.org/conferences/?conf=gamifir2016

We aim for an interactive and dynamic full-day workshop that will be a mix of keynotes, paper and demo presentations as well as discussion sessions.

Previous GamifIR


Program Committee

  • Omar Alonso, Microsoft Research (USA)
  • Raian Ali, Bournemouth University (UK)
  • Leif Azzopardi, University of Glasgow (UK)
  • Jon Chamberlain, University of Essex (UK)
  • Sebastian Deterding, Northeastern University (USA)
  • Carsten Eickhoff, ETH Zurich (CH)
  • Christopher G Harris, State University of New York (USA)
  • Hideo Joho, University of Tsukuba (JP)
  • Till Plumbaum, TU Berlin (GER)
  • Ashok Ranchod, University of Southampton (UK)
  • Thomas Springer, TU Dresden (GER)
  • Susanne Strahringer, TU Dresden (GER)
  • Albert Weichselbraun, University of Applied Sciences Chur (CH)
  • Lincoln Wood, The University of Auckland (NZ)

Organizing Committee

  • Frank Hopfgartner, University of Glasgow (United Kingdom)
  • Gabriella Kazai, Semion Ltd. (United Kingdom)
  • Udo Kruschwitz, University of Essex (United Kingdom)
  • Michael Meder, TU Berlin (Germany)

Important Dates

  • Submission deadline: May 29, 2016
  • Notification date: June 19, 2016
  • Camera-ready due: July 3, 2016
  • Workshop date: July 21, 2016

ECIR 2016 (Padua, Italy)

Author: Andreas Lommatzsch

The European Conference on Information Retrieval 2016 was held in Padua (Italy) from March 20th to March 23rd, 2016. The topics of the conference cover a wide spectrum ranging from classical IR algorithms evaluated on static dataset, to Question Answering, to Evaluation Methodologies and Machine Learning.

The opening keynote was given by Jordan Boyd-Graber (University of Colorado, USA). The interesting talk "Opening up the black box: Interactive Machine Learning" discussed the role of components built based on AI algorithm in the society. In order to trust in these systems, transparent machine learning algorithms are needed, especially for complex tasks. Improving the transparency and the provided explanation is more important that improving the precision for providing the optimal support for decisions made by humans.

On Tuesday Emine Yilmaz (University College London, UK) gave the keynote titled "A Task-based Perspective to IR". The talk outlined that IR should focus on task trees. In contrast to simple list of documents ranked by assumed relevance, task trees are powerful tools supporting users in solving complex problems. The talk summarized the state-of-the art and outlined current trends in this domain.

On the third day of the conference, I attended the Industry session. Domonkos Tikk (Gravity R&D, Hungary) gave the keynote "Lessons learned at Building Recommender Services in Industry scale". The very interesting talk explained the challenges of adapting algorithms for real-life applications and strategies for handling the technical complexity of algorithms.
In the subsequent session Jonas Seiler (plista), Daniel Kohlsdorf (XING) and I presented the talk "Get on with it – Recommender system industry challenges move towards real-world, online evaluation". In the presentation we suggested approaches for applying more realistic evaluation settings. Starting with the evolution of the evaluation approaches in IR and recommender systems we outlined the challenges evaluating real-life systems taking into account not only the precision of the results but also technical aspects and business models. We suggested a closer cooperation between academia and industry to their mutual benefit e.g., by participating in challenges organised by industry and academia. The evaluation should consider heterogeneous data as well as, streamed data, and focus on multi-dimensional metrics taking into account the needs of users and the service providers. The RecSys Challenge 2016 as well as the CLEF NewsREEL-Challenge offer interesting opportunities for evaluating own algorithms in real-life scenarios.

The venues of the conference have been impressive: At the first day, the conference talks took place in the Botanical Garden. The second and third were held in the historical university building next to the town hall. In the "Aula Magna" Galileo Galilei gave lectures in the 16th century.

The following photos give a visual impression of the ECIR 2016.

CrowdRec Contribution to SIREMTI Workshop / MobiCASE

Author: Benjamin Kille

The 7th EAI International Conference on Mobile Computing, Applications and Services took place on 12-13 November in Berlin. The Workshop on Situation Recognition by Mining Temporal Information was held as part of the event.

We contributed a paper in collaboration with Fabian Abel (XING) and Balázs Hidasi (Gravity R&D). “Using Interaction Signals for Job Recommendation” describes and evaluates recorded interactions between professionals and job offers. In addition, a user inquiry delivers the ground truth on how relevant professionals perceived suggested job offers. We investigated three types of events: clicking, bookmarking, and replying. Professionals can reply to suggested jobs in three ways:

  • messaging the recruiter
  • request additional information
  • follow a link to an application form

Our data show that replying correlates with high relevant suggestions. Bookmarks correlate predominantly with medium relevant suggestions. Individual clicks have a wide-spread distribution. They fail to differentiate highly relevant from irrelevant jobs. As the professional repeatedly click on suggested jobs, the correlation tends to more relevant jobs. slides

The conference provided a diverse set of talks. Topics included the Internet of Things, Wireless Mesh Networks, 3D-Printing, and QR Codes. Besides our contribution, there were two papers discussing recommender systems. Nadeem Jamali (University of Saskatchewan) introduced CSSWare, a framework to use crowd-sourced data to enhance recommendation. Mouzhi Ge (Free University of Bozen-Bolzano) described how logging devices enable intelligent dining recommendations. Their work highlighted the connection between life logging and decision support in the domain of nutrition and eating.

The diverse mixture of research provided many chances for interesting conversations during lunch an the breaks.