Overview
Commercial providers of information access systems (such as Amazon or Google) usually evaluate the performance of their algorithms by observing how large numbers of their customers interact with different instances of their services. Unfortunately, due to the lack of access to large-scale systems, university-based research is struggling to catch up iq option with this large-scale evaluation methodology. NewsREEL, short for News Recommendation Evaluation Lab, aims to bridge this “evaluation gap” between Academia and Industry.
NewsREEL is organised as a campaign-style evaluation lab of CLEF 2017 and addresses the following information access task:
Whenever a visitor of an online news portal reads a news article on their side, the task is to recommend other news articles that the user might be interested in.
NewsREEL offers two tasks to study this use case. The first task, NewsREEL Live, implements the idea of a “living lab” where the provider of a recommendation service provides access to its infrastructure and user base. The second task, NewsREEL Replay, replays a live setting using the recommender system reference framework Idomaar.
By providing this service for millions of users, the recommendation scenario requires solutions to significant research challenges, such as processing information in real-time, handling vast amount of data, and providing suitable recommendations. By providing access to the infrastructure of a company, we offer professionals and students the opportunity to develop skills that are in high demand in industry, while at the same time allowing them to familiarize themselves with the academic practice of evaluation of information access systems.
Please follow @CLEFNewsREEL for further updates.
See also: