Nick Palmer is probably itching to say this.
We have left out a key point in this discussion. Even though
consensus is secondary evidence, and not as satisfactory as direct
experimental data, there are times when we must depend upon it. In
the case of global warming, we cannot do a planetary scale experiment
to find out if the global warming hypothesis is true, so we must
settle for computer models, consensuses among experts, and so on.
(To be more exact we are already doing a planetary scale experiment
but that is not a good idea, and besides if it turns out badly enough
we will not survive to take data, so it is an unsatisfactory
experimental technique.)
There are many other instances in which we cannot do a confirmation
experiment at some stage, and we must depend upon consensus. Suppose,
for example, in the early stages of a research project a committee
has assembled to decide whether to fund experiment A or B. They have
to use their best judgment and go by consensus because no one knows
in advance which is better. You have to do them both to find out for
sure. If you already knew there would be no need to call the committee meeting.
In some instances we must rely upon reputation rather than consensus.
In the initial stages of cold fusion in March 1989, most decisions to
go ahead with replication attempt were predicated upon Fleischmann's
reputation. Reputation is a poor method of judging an experiment. It
is even worse than consensus. But it is better than nothing.
- Jed
