RMIT CIDDA IR at the TREC 2022 Fair Ranking Track

Abstract

The quality of articles in Wikipedia greatly depends on the level of editorial effort that is deployed to maintain them. In an ideal world, every document that requires editing would get the attention needed by Wikipedia editors to improve its quality. Unfortunately, the demand for editorial effort is substantially higher than the available editorial resources. To meet this limitation, Wikipedia coordinators/editors use search engines to find the Wikipedia documents that will be edited. Articles and topics in Wikipedia have different characteristics in terms of protected attributes. If the search engines used in the editorial process do not consider these attributes, different societal groups may receive disparate outcomes, eventually leading to negative social and economic impact. The Fair Ranking Track at TREC 2022 aims to address the problem of providing fair rankings of Wikipedia articles for coordinators/editors. This report describes the participation of the RMIT CIDDA IR group in Task 1 of the track, which consists of providing a fair ranking of Wikipedia articles when multiple protected attributes need to be considered. We addressed this challenge by proposing a three-stage approach that includes ad hoc retrieval, explicit Search Results Diversification (SRD), and ranking fusion. We also explored the use of a multi-criteria decision-making method (Analytic Hierarchy Process, AHP) as a mechanism to allow stakeholders to have control of the way that different attributes could be combined in the fusion stage.

Citation

If you find this paper useful, please cite it using the following BibTeX:

@ARTICLE{cherumanal2022rmit,
    TITLE = {RMIT CIDDA IR at the TREC 2022 Fair Ranking Track},
    AUTHOR = {CHERUMANAL, SACHIN PATHIYAN and ALAOFI, MARWAH and ALTALHI, REHAM ABDULLAH and NAGHIZADE, ELHAM and SCHOLER, FALK and SPINA, DAMIANO}
    PUBLISHER = {TREC}
    YEAR = {2022}

}