Call for Papers

The Machine Learning Reproducibility Challenge (MLRC 2023) is an unique, online conference which encourages the community to investigate the reproducibility, replicability and generalisability of published claims in top conferences in the literature. We invite submissions which investigate the recently published claims, add novel insights to them, and enable reproducible research, spanning various topics in the ML literature. Submissions must be first accepted at TMLR to be considered in the MLRC 2023 Proceedings. Please read the author guidelines and submission guidelines from TMLR to get the submission format and to understand more about the reviewing process. Please read our announcement blog post for more motivation, retrospectives and roadmap for the challenge.

Scope

We invite thorough reproducibility studies, including but not limited to:

  • Generalisability of published claims: novel insights and results beyond what was presented in the original paper. We recommend you choose any paper(s) published in the 2023 calendar year from the top conferences and journals (NeurIPS, ICML, ICLR, ACL, EMNLP, ICCV, CVPR, TMLR, JMLR, TACL) to run your reproducibility study on.
  • Meta-reproducibility studies on set of related papers.
  • Research on tools enabling reproducible research.
  • Meta analysis on the state of reproducibility in various subfields in Machine Learning.

Important Dates

  • Challenge goes live: October 23, 2023
  • Submit to TMLR OpenReview: https://openreview.net/group?id=TMLR
  • Deadline to share your intent to submit a TMLR paper to MLRC: February 16th, 2024 at the following form: https://forms.gle/JJ28rLwBSxMriyE89. This form requires that you provide a link to your TMLR submission. Once it gets accepted (if it isn’t already), you should then update the same form with your paper camera ready details.
  • We aim to announce the accepted papers by May 31st, 2024, pending decisions of all papers.

Camera Ready Process

  • After you have updated the form with your accepted TMLR paper, it will finally undergo a light AC review to verify MLRC compatibility.
  • We will publish a proceedings booklet post announcement of all decisions.
  • Accepted papers will be featured in our website along with 5-min companion videos.

For query regarding MLRC 2023, contact us at mlrc-2023@googlegroups.com.

Koustuv Sinha

Program Chair, MLRC 2023

Last updated on Oct 22, 2023