[NeurIPS’22] Chasing reviewers

as some of you may have noticed, i was one of the program chairs of NeurIPS’22 which just ended last Friday (December 9 2022). it was a two-week-long conference with the first week being in person in New Orleans which was followed by the virtual week. program chairs were mostly tasked with running the review process for the main track of the conference and inviting keynote speakers, and there were other organizing committee members who have taken care of various other aspects of the conference, including expos, workshops, tutorials, datasets and benchmark track, social events, affinity workshops and many more, under the guidance of our amazing general chairs, Shakir Mohamed and Sanmi Koyejo.

a major part of running the reviewing process is to ensure all the reviews, meta-reviews, decisions as well as decision agreements are received in time, in order to ensure that we find a set of quality papers to be presented at the conference and that the authors of these accepted papers as well as participants are given enough time to prepare their travels to the conference. in a sense, everyone, including program chairs ourselves, agrees to serve a role, either as a reviewer, area chair or senior area chair, at the time of invitation, and we might naively expect all to stick to the timeline. it is however not the case due to the sheer scale of the conference’s main track, with more than 13,000 submitted abstracts and more than 10,000 reviewers we recruited. what is the chance that every single reviewer is fully available without any personal or professional emergencies over the summer, which is the period over which NeurIPS reviewing happens? if we assume 0.01% of personal/professional emergency for each individual reviewer, the chance that every one is available fully over this period is less than 40% …

now of course on top of that, we are all humans and simply make mistakes by for instance forgetting to put various deadlines on our calendars or simply over-committing ourselves. these mistakes can be however mitigated to some degree by reminders, or at least that was my thought back this summer (2022).

as part of this effort of politely but strongly reminding reviewers as well as area chairs of upcoming deadlines, i decided to finally benefit from a reasonably large number of followers i have on twitter (as of Dec 12 2022, i have 42.5k followers). who knew i would ever use twitter for my own benefit (and i want to say, for the community’s benefit)? but, the time had finally arrived …

i decided to piggy-back on people’s liking of memes on Twitter and started to post quite regularly NeurIPS’22 reviewing memes. It started on Jun 23 2022 and then continued until July 14 2022. here, i’ll list all of them for you to easily see how quickly my mind has sprawled into a dark abyss over time … i am in fact unsure if i’ve ever gotten out of this dark abyss i fell through …

… with even all these tweets, we failed to collect all necessary reviews in time …

Leave a Reply