Stanford Digital Library Project

SIDL-WP-1995-0025

LIRA II: Experimental Design

Marko Balabanovic

marko@cs.stanford.edu

Abstract: In this report I will summarize the shortcomings of the evaluation methods used for the previous LIRA experiments. I will then give a brief discussion of some measures used by the IR community: both the traditional relevance-based measures, and some newer ideas on using user preferences. Finally I will outline the design for experiments to be performed with the new LIRA II system.

Note: Papers in this series are in development and are not in a final form for publication or general dissemination. They are subject to change. Please do not quote or further distribute them without explicit permission from the authors.

This paper was created on: 11/22/95 and last revised on:11/22/1995

Author's Comments: First cut experimental design for upcoming user studies, comments encouraged.

Status: PRIVATE

Click here to see the full text of SIDL-WP-1995-0025 (PS)