We welcome the submission of research papers and abstracts from the broader ML community describing original work that has not been submitted or currently under review, has not been previously published nor accepted for publication elsewhere, in any other journal or conference. In particular, this work may touch on one or more of the following aspects:

  • Unexpected negative results or anomalies: ideas that do not provide expected results, yet authors are able to explain why, bringing an interesting closed-form piece of knowledge to the community.
  • Papers that are “stuck” yet contain beautiful/elegant ideas. Authors should argue why the idea is of interest, rigorously describe the analysis, and include a self-critique.
  • Criticism of and alternatives to default or standard practices (e.g., current evaluation metrics).
  • Meta-research on the role of “beauty” or negative results in broader ML research (including statistics, data science, AI, and application areas).

If you think your paper falls within the remit of the workshop but does not clearly fit one of these tracks, please contact the organizers at cant.believe.it.is.not.better@gmail.com

Formatting instructions

Submissions should be 4 pages long at most (not including references) and submitted via OpenReview platform by September 17th 2021. The Appendix can be unlimited, but note that reviewers might only read the main text. Accepted authors will be invited to participate in a poster session at the workshop. These submissions are non-archival, however reviewers will nominate exemplar papers for submission in PMLR.

Submissions will be double-blind, so you should submit an anonymous preprint.

You must format your submission using the provided LaTeX style files.

The maximum file size for submissions is 50MB. Submissions that violate the provided style (e.g., by decreasing margins or font sizes) or page limits may be rejected without further review.

How to submit

Submissions to the workshop will be handled through our OpenReview site.

Main deadline: September 17th September 21th 23:59 Anywhere on Earth. Accept/reject notification will be sent out by October 15th October 17th.

Reviewing criteria

Reviewing and awards: Reviewers will not be asked to assess “beauty”, but rather to evaluate whether authors have identified a gap between intrinsic and extrinsic value, broadly speaking, and have clearly explained their ideas. Reviewers will reward novelty and elegance, thorough understanding, transparency, and reproducibility, rather than high empirical performance. Reviewers will also nominate papers for two awards: the “Entropic Award”, for most surprising gap or negative result, and the “Didactic Award”, for most well-explained and pedagogical paper. Reviewers will also nominate papers with exemplary scientific rigor and high-quality research process for publication in PMLR.

We will be selecting for submissions that provide interesting insights on unexpected results. Morevover, we expect all submissions to follow the guidelines of basic courtesy and respect.

Submissions will be evaluated on the following criteria:

  • Clarity of writing
  • Rigor and transparency in the scientific process
  • Vulnerability and honesty in discussion, particularly if the submission is by the original author
  • Quality of discussion of limitations
  • Significance of new insights