BMJ and Findzebra use cases in a computable format

We formatted these resources to generate a set of relevance judgements for the evaluation of Clinical Decision Support Systems in the field of Rare Diseases. We merged two originally published BMJ and IJMI data sets in a convenient and interoperable format following TREC standards (QREL). Such a benchmark is appropriate to assess the effectiveness of the WeIRD diagnosis support system, see

These data are compiled based on [Dragusin 2013] and [Tang and Ng 2006].

  • Dragusin, R., Petcu, P., Lioma, C., Larsen, B., Jørgensen, H. L., Cox, I. J., ... & Winther, O. (2013). FindZebra: a search engine for rare diseases. International journal of medical informatics, 82(6), 528-538. (see also, the FindZebra search engine for rare diseases)

  • Tang, H., & Ng, J. H. K. (2006). Googling for a diagnosis—use of Google as a diagnostic aid: internet based study. Bmj, 333(7579), 1143-1145.