Reviewer Instructions

Important Dates

Reviewing Instructions

Two-minute overview

 The ECCV 2020 review process is double blind and confidential.

• Keep the paper confidential. Do not distribute it, except to another person in your group to help you with the review.
• Do not reveal your identity to the authors

 Criteria

• Rate a paper by the value of its scientific contributions (valuable for at least 2% of the computer vision community, not necessarily yourself). Good benchmark numbers can be an indicator for a valuable contribution, but they are neither necessary nor sufficient for a valuable paper.
• Give enough reasons and information for your rating so that authors and ACs can understand it. “Not new” must be backed by concrete references.
• Papers that are not accepted at a peer-reviewed venue do NOT count as prior work. Do not ask authors to compare to such work. Do not reduce your novelty rating because of such concurrent work.

 Style

• Do not be emotional in your review and watch your tone. Make it useful for the authors and the area chairs. Read over your full review before you finalize it.

 Logistics

• Update your profile in OpenReview before the submission deadline.
• Check your papers for conflict of interest within 2 days after assignment.
• Submit all your reviews by the reviewing deadline. Delays severely affect the timeline of the reviewing process and are not acceptable.
• Respond to questions by the area chair in the discussion phase.
• Submit all your final recommendations by the communicated deadline.
  

 Perks

• Free registration will be awarded to the best reviewers (as rated by the Area Chairs). Reserved registration slots will be offered to a larger group of best reviewers.

If you have further questions or doubts, read the detailed instructions below.

Check your papers immediately to avoid a conflict of interest

As soon as you get your reviewing assignment, please go through all the papers to find cases where:

1. There is a conflict of interest (e.g., a paper authored by your recent collaborator from a different institution, or a concurrent work with the same contribution as the work you submitted yourself)
2. You are unable to provide reasonable review to the paper. However, do not give up just because the paper is not of your liking or because it does not perfectly match your area of expertise. Due to various hard constraints, it is unavoidable that not all assignments are optimal, and reviewers are asked to make a reasonable effort to review the assigned papers.
   
   If either of these issues arise, please let us know right away by emailing the Program Chairs.

ECCV makes every effort to avoid conflicts in the review assignments, but errors can occur. If you think you have a conflict of interest with a paper you are reviewing, you are required to contact the Program Chairs to resolve the matter.

Conflicts of interest include (but are not limited to) situations in which:

• You work at the same institution as one of the authors.
• You have been directly involved in the work and will be receiving credit in some way. 
• You suspect that others might see a conflict of interest in your involvement.
• You have collaborated with one of the authors in the past three years. Collaboration is usually defined as having written a paper or having a joint grant.
• You were the MS/PhD advisor of one of the authors or the MS/PhD advisee of one of the authors. Most funding agencies and publications typically consider advisees to represent a lifetime conflict of interest. ECCV has traditionally been more flexible than this, but you should think carefully before reviewing a paper you know to be written by a former advisee, especially a recent one.

Rebuttal

Authors will be given an opportunity to answer your criticisms in a rebuttal. In your review, you can ask the authors to provide specific information in the rebuttal, such clarifications on the proposed algorithm and on the experimental results. However, refrain from asking brand new experiments. The rebuttal is not a major revision of the paper, nor is fair to ask to the authors to perform complex tasks in the short time available for the rebuttal. If you think that a crucial experiment is missing and that the paper cannot be accepted without it, you can reject the paper; however, do not ask the authors to perform this experiment in the rebuttal. The Area Chair can discount your requests if they are unreasonable.

Look for innovation, not just benchmarks

Look for what is good or stimulating in the paper. Minor flaws can be corrected and shouldn’t be a reason to reject a paper. ECCV as a conference is looking for new ideas. We recommend that you embrace novel, brave concepts, even if they have not been tested on your favourite datasets, as long as solid scientific conclusions can be drawn from the experiments. For example, the fact that a proposed method does not exceed the state-of-the-art accuracy on an existing benchmark dataset is not grounds for rejection by itself. The method may highlight a weak spot of an established benchmark and may have other scientific evidence that supports the claims of the paper. Benchmarks are a popular way to support claims, but there are other ways. Each paper that is accepted should be technically sound, should prove claims using scientific principles, and must contribute in some way to scientific progress in the field.

Be specific and provide evidence

Please be specific and detailed in your reviews. For example, simply saying “this is well known” or “this has been common practice in the industry for years” is not sufficient: cite specific publications, including books, or public disclosures of techniques and explain the concrete overlap with the reviewed paper. Always explain your rating. Your discussion, sometimes more than your score, will help the authors, fellow reviewers, and Area Chairs understand the basis of your opinions, so please be thorough.
Your reviews will be returned to the authors, so you should include specific feedback on ways the authors can improve their papers or their research. A harshly written, emotional review will be disregarded by the authors, regardless of whether your criticisms are valid. State you criticisms clearly and concisely to help the Area Chair make a decision based on your review, but also put yourself in the mind-set of writing to someone you wish to help, such as a respected colleague who wants your opinion on a concept or a project to help the authors improve their work. 
A thoughtful review not only benefits the authors, but may benefit you as well. Remember that your reviews are read by other reviewers and especially the Area Chairs, in addition to the authors. In contrast to the authors, they will know your name. Being a helpful reviewer will improve your reputation in the research community. In contrast, returning an uninformative, flawed, late review will damage your reputation.

Submission guidelines

ECCV has clear policies for submission, including double submission and plagiarism. Please check that the authors adhere to them and see the Author Guidelines if you are in doubt.
On the other hand, make sure you do not “invent” policies beyond the ones explicitly adopted by ECCV. For instance, ECCV does not have a policy that a dataset or code must be made publicly available. While this is encouraged, there are many valid reasons why this is not always possible. Thus, it is unfair to reject a paper for the reason that it contains no promise to make a public release of the data or code. The actual issue you should check is whether the scientific results in the paper are reproducible, which is easier if code is provided.

ArXiv papers

Authors are asked to take reasonable efforts to hide their identities, including not listing their names or affiliations and omitting acknowledgments during review. Publishing technical reports and arXiv papers, however, is allowed. Do not actively search for the paper on arXiv to keep an unbiased view as much as possible. 
In line with common practice in the community, arXiv papers are not considered prior work since they have not been peer reviewed. Therefore, you should review your ECCV papers independently as if arXiv papers did not exist. Citations to these papers are not required and failing to cite or beat performance of arXiv papers are not grounds for rejection. Please read the FAQ below for guidelines on handling arXiv papers.

Protect ideas

The ECCV 2020 review process is double blind and confidential. An ECCV submission does not constitute a public disclosure. As a reviewer for ECCV, you have a responsibility to protect the confidentiality of the ideas represented in the papers you review. Breaching confidentiality can have serious consequences for the authors and their organisations, such as loss of intellectual property if a patent is pending.
In particular:

• Do not show the paper or supplementary material to anyone else, including colleagues or students, unless you have asked them to write a review, or to help with your review.
• Do not discuss the ideas or results in the paper with any non-reviewer.
• Do not use or build on the ideas in your own work.
• After the review process is complete, you must destroy all copies of the paper and erase notes or code you have written to evaluate the ideas in the paper, including results produced by any such code.

Further guidelines

• Do not ask the authors to cite your papers unless this is clearly justified. Be aware that the Area Chair can see your name.
• Before you claim a paper is out of scope, carefully check the Call for Papers, clearly explain why, and try to suggest a better venue for it.
• Avoid referring to the authors by using the phrase “you” as it may sound confrontational; use instead “the authors” or “the paper.”

Frequently asked questions

Is there a minimum number of papers I should accept or reject?
No. Each paper should be evaluated in its own right. Do NOT assume that your stack of papers necessarily should have the same acceptance rate as the entire conference ultimately will. You may have a stack of particularly good or bad papers.

Can I review a paper I already saw on arXiv and hence know who the authors are?
Yes. See next bullet below for guidelines.

How should I treat papers for which I know the authors?
Treat them like papers of which you do not know the authors. Reviewers must assess a paper only based on its content and not based on the authors.

How should I treat arXiv papers?
ArXiv papers are not considered prior work since they have not been peer reviewed. Therefore, you should review your ECCV papers (largely) as if the arXiv papers did not exist. For example:

• Do not reject a paper because it has similar ideas on one that appeared on arXiv. Only if you think the arXiv paper was plagiarized, notify this issue in the confidential comments to the AC.
• Do not suggest rejection for not citing an arXiv paper.
• Do not regard arXiv as a standard for the state of the art. It is not, because it is not peer reviewed. Hence, do not accept/reject a paper solely because it performs better/worse than something on arXiv.

On the other hand:

• You can suggest to acknowledge and be aware of something on arXiv.
• You can decline to acknowledge something on arXiv (because it has not been peer reviewed and so may not be right).