Review Guidelines

From MCH2022 wiki
Jump to navigation Jump to search

Review guidelines

Technical process

The review process is divided into two phases: Initial and Decisions.

In the Initial phase, Team:Content score the reviews on a points scale of 1 - 7 and (if wanted) can add a note explaining their choice

  • make sure you (re)read the CfP before you start reviewing, so that you understand:
       - what categories there are
       - what is requested of presenters to submit
       - what types of submissions are available
       - to reacquaint yourself with the total feel for the event


       - try to form an opinion as you read, keeping an open mind
       - try not to be influenced too much by name or reputation of submitter (both positive or negative)
       - see if it fits a category, or fits with the overall feel of the event
       - if it doesn’t fit, do they have a good story for why they should present anyway?
       - if they describe the structure of the presentation?
       - if they explain why this is interesting for the audience?
       - if they explain what the audience will learn?
       - would there be people interested in this? try to keep an open mind, and picture the wide audience for the event
       - if you feel that the submittter is capable of presenting a convincing story? (e.g. based on earlier presentations, a well structured submission, etc.)
       - try not to use the information about submissions for your own gain

Please note, this is not a checklist for talk submission, but a list of things reviewers keep in mind during the review process.

These reviews - and indeed, other communications about speakers / talks / workshops - are considered confidential information to anyone outside of Team: Content.

At one point, the flow of papers coming in will be very rapid and we will have meetings to discuss the outstanding inputs. It will become very important to review all the talks before the meetings so we know we can discuss the same things. Be prepared, this is a considerable time sink.

During the Initial Phase, some talks will be so outstanding that they will (with unanimous consent of Team: Content) be accepted early. These will be given to Team: Communications to promote the conference (and the CFP) with.

Team:Content members get to use a "Suspensive Veto" a talk or force through a talk (or a number of them per person), though this "Suspensive Veto" or Pass will always be subject to discussion in Team:Content. To be very clear it is absolutely not the idea that any one member of Team:Content can push through or cancel a talk on their own. Please look up the term "Suspensive Veto" for more clarity :)

In the Decision phase, Team:Content will choose the highest scoring talks, count them and confirm as many as there are slots, or as many as are worth confirming if there are less talks than slots. There will be some discussion regarding the subjective overscoring of some talks or impassioned pleas for raising scores of talks team members may have missed the finer nuances of. Also finance requests are to be taken into account - it may be that we simply don't have enough budget for certain speakers or that we simply don't want to spend the requested amount of budget on a certain speaker / workshop / talk.

After the Decision phase, the talks will be confirmed and the speakers will be emailed.

Finally, Team:Content will have a "sticky note" session where we print all the talks with ratings and by subject (maybe) on small notes and try to roster them in for each day of the conference. This will take a few full weekend days (from early morning to late at night).


As members of Team:Content we represent a wide variety of backgrounds, interests, philosophies, hobbies, professional skillsets, etc. We are a diverse team of individuals, each representing some part of the set of visitors to this set of conferences and hopefully also a set of visitors that doesn't traditionally come to these conferences. This way we keep fresh ideas coming in, a fresh pool of talent to entertain the conference guests and a fresh pool of visitors to grow "the community" (whatever that exactly is) with.

This means that filling the available speaking and workshop slots with content that We Think Is Interesting is automatically catering to the wishes of "the community". It also means that when someone on Team:Content suggests something, this may be far removed from another team members' everyday experience. Remember that each team member is trying to enrich the experience for all and so please take their suggestions seriously and openly, even if you don't maybe initially understand it. Having said that, not every opinion always bears close examination. We will disagree. A Lot. And that's a good thing - because as long as people are given the room to explain their viewpoint, this will make the whole process more robust.

Given the need for open and (sometimes brutal) honesty in these discussions, emails and reviews Team:Content members need to be assured that their thoughts in this matter are kept confidential to protect from repurcussions against them or the speakers being considered.

Vote your conscience after having tried to see things from another perspective. In the end the team members are putting in the hours during meetings and are expected to give all talks their serious attention, even if they have funny titles or even if you don't immediately understand them. As long as you can motivate your decision, your decision is valid because of who you are.

A word on integrity. The only people who decide what goes on the official program is Team:Content. This means that the team has the right to refuse any content not deemed fit, including content submitted by sponsors, projectleiding or other teams. The team may of course decide that these are special cases and have priority in the review process, so that they can resubmit more fitting content with a larger chance of passing the review process, as the team may decide with other promising submissions.

As the conference nears, the amount of participative time required in the team will grow exponentially. The speakers meetings will be more intense and the sticky notes meetings will require everybody's input.

During the conference itself, Team:Content will be required to help the Angels and Speakerdesk with catering for the needs of speakers, ensuring the budget is being correctly allocated, ensuring the stages and workshop rooms are prepared, coordinate with the AV team and much more, re-schedule or cancel talks.