Program
Tuesday November 11
Time | Event |
08:30 - 09:30 | Registration |
09:30 - 09:45 | Welcome |
09:45 - 11:15 |
Session 1: Domains (chair: Paul Buitelaar)
|
11:15 - 11:45 | Coffee |
11:45 - 13:00 | Panel: Professional Search in a Modern World (chair: Kalervo Järvelin) |
Panel members: Fabio Crestani, Gabriella Pasi, Christian Boitet, Stephane Marchand-Maillet, Elaine Toms | |
13:00 - 14:00 | Lunch |
14:00 - 15:00 | Keynote: "Don’t Hurt Them: Learning to Rank from Historical Interaction Data" by Maarten de Rijke) (chair: Paul Buitelaar) |
15:00 - 15:30 | Coffee |
15:30 - 17:00 |
Session 2: Systems and algorithms (chair: Stephane Marchand-Maillet)
|
18:30 - 22:00 | Conference dinner at Restaurant SULT |
Wednesday November 12
Time | Event |
08:30 - 09:00 | Registration |
09:00 - 10:30 | MUMIA Management Committee Meeting & Evaluation –- Reporting 1 (chair: Michail Salampasis) |
Action Chair and Grant Holder presentations Presentations from Working Group leaders | |
10:30 - 11:00 | Coffee |
11:00 - 12:20 | MUMIA Management Committee Meeting & Evaluation –- Reporting 2 (chair: Andreas Rauber) |
STSMs, IR/NLP/MLT integration, Standards & Protocols, Dissemination, Training Schools, Industry and Organisation Impact, Early Stage researchers, and Case Studies. | |
12:20 - 13:00 | Discussion and Questions |
13:00 - 14:00 | Lunch |
14:00 - 15:30 |
Session 3: Design and evaluation (chair: Allan Hanbury)
|
15:30 - 16:00 | Coffee |
16:00 - 17:00 | Panel: The Next Steps after the MUMIA COST Action (chair: John Tait) |
Panel members: Fernando Loizides, Mihai Lupu, Laurentiu Vasiliu, Mike Salampasis |
You can find the PDF version of the agenda of the MUMIA Management Committee Meeting & Evaluation here.
Keynote
Don’t Hurt Them: Learning to Rank from Historical Interaction Data Maarten de Rijke (University of Amsterdam)
One of the main advantages of online evaluation schemes is that they are user-based and, as a result, often assumed to give us more realistic insights into the real system quality than off-line methods. This is also one of their main disadvantages: comparing two rankers online requires presenting users with result lists based on those rankers and observing how users interact with them. New rankers may perform sub-optimal and hence hurt the user experience. Can we use or re-use historical data, collected from user interactions with a production system, to assess or optimize new alternative rankers? This question has increasingly gained interest in the past few years. In the talk I will contrast several proposals for learning from historical interaction, based on importance sampling, random buckets, and a Bayesian approach based on explicit user models.
This is based on joint work with Artem Grotov, Katja Hofman, Damien Lefortier, Anne Schuth, Shimon Whiteson.
Maarten de Rijke is full professor of Information Processing and Internet in the Informatics Institute at the University of Amsterdam. He holds MSc degrees in Philosophy and Mathematics (both cum laude), and a PhD in Theoretical Computer Science. He worked as a postdoc at CWI, before becoming a Warwick Research Fellow at the University of Warwick, UK. He joined the University of Amsterdam in 1998, and was appointed full professor in 2004.
De Rijke leads the Information and Language Processing Systems group, one of the world's leading academic research groups in information retrieval. During the most recent computer science research assessment exercise, the group achieved maximal scores on all dimensions. His research focus is on intelligent information access, with projects on self-learning search engines, semantic search, and social media analytics.