The Academic Curmudgeon
Jerzy Neyman, one of the founding fathers of
modern statistics, said:
``... it is my experience that whenever a generally decent fellow is
asked to act as an anonymous referee, he is apt to acquire hateful
qualities: presumptuousness, quarrelsomeness, and bossiness.''
It is my experience as an Associate Editor (14 years) and an author
(17 years) that Neyman was right. Many colleagues and friends agree
with me that our peer review system is rife with unfair, unhelpful and
downright hostile refereeing.
Two questions come to mind: Why is this? and Does it Matter?
The second question is easy to answer: it matters. Dealing with nasty
referees is time consuming, draining and wastes valuable research and
teaching time. It discourages young researchers and embitters them.
Today's mistreated young researchers are tomorrow's nasty referees. It
is a self-perpetuating problem.
The first question will require future writings to address. But as a
first quick pass at the question, let me say that many referees reject
papers for the wrong reasons. Here is a summary of common reasons for
rejecting papers:
Good Reasons For Rejecting a Paper:
1. The results are incorrect (unfixable, critical errors).
2. The results are not new.
Bad Reasons For Rejecting a Paper:
3. The referee doesn't like the paper.
4. The referee doesn't like the author's approach.
Tricky:
5. The contribution of the paper is too small.
Numbers 3 and 4 are bad because they are based on the taste of the
referee which is far too subjective. Number 5 is problematic. True,
we don't necessarily want top journals publishing every small idea
that occurs to someone. The problem is this: almost all research,
including good research, is incremental. The idea that most papers in
top journals are breakthough papers is fantasy. What is too
incremental to publish is highly subjective.
I think the basic problem is that most referees have the wrong
view of the purpose of publishing. Ideally, publishing is about
disseminating knowledge. It should not be regarded as admittance to
a high and mighty priesthood.
modern statistics, said:
``... it is my experience that whenever a generally decent fellow is
asked to act as an anonymous referee, he is apt to acquire hateful
qualities: presumptuousness, quarrelsomeness, and bossiness.''
It is my experience as an Associate Editor (14 years) and an author
(17 years) that Neyman was right. Many colleagues and friends agree
with me that our peer review system is rife with unfair, unhelpful and
downright hostile refereeing.
Two questions come to mind: Why is this? and Does it Matter?
The second question is easy to answer: it matters. Dealing with nasty
referees is time consuming, draining and wastes valuable research and
teaching time. It discourages young researchers and embitters them.
Today's mistreated young researchers are tomorrow's nasty referees. It
is a self-perpetuating problem.
The first question will require future writings to address. But as a
first quick pass at the question, let me say that many referees reject
papers for the wrong reasons. Here is a summary of common reasons for
rejecting papers:
Good Reasons For Rejecting a Paper:
1. The results are incorrect (unfixable, critical errors).
2. The results are not new.
Bad Reasons For Rejecting a Paper:
3. The referee doesn't like the paper.
4. The referee doesn't like the author's approach.
Tricky:
5. The contribution of the paper is too small.
Numbers 3 and 4 are bad because they are based on the taste of the
referee which is far too subjective. Number 5 is problematic. True,
we don't necessarily want top journals publishing every small idea
that occurs to someone. The problem is this: almost all research,
including good research, is incremental. The idea that most papers in
top journals are breakthough papers is fantasy. What is too
incremental to publish is highly subjective.
I think the basic problem is that most referees have the wrong
view of the purpose of publishing. Ideally, publishing is about
disseminating knowledge. It should not be regarded as admittance to
a high and mighty priesthood.
posted by larrywasserman | 8:09 AM
|
9 comments
