3.3 Popperian Falsificationism Next we turn to the falsificationist ideas of Karl Popper. According to this theory of the scientific method, we test a hypothesis by deducing from it a prediction that can be tested in an experiment. If the prediction fails to hold in the experiment, then the associated hypothesis is said to be falsified and must be rejected. Thus Popperian falsificationism requires a scientist to hold a hypothesis tentatively, to explore and highlight the ways in which the hypothesis might break down, to uncover and scrutinize evidence contrary to the hypothesis rather than discarding or suppressing such evidence, and in general to avoid exaggeration or overstatement of the evidence supporting the hypothesis. Perhaps the most forceful statement of this view of science was given by Richard Feynman in the quotation at the beginning of this article. According to Woodward and Goodstein, there are also serious deficiencies in Popperian falsificationism as a general theory of good scientific method: One of the most important of these is sometimes called the Duhem-Quine problem. We claimed above that testing a hypothesis H involved deriving from it some observational consequence O. But in most realistic cases such observational consequences will not be derivable from H alone, but only from H in conjunction with a great many other assumptions A (auxiliary assumptions, as philosophers sometimes call them). ... It is possible that H is true and that the reason that O is false is that A is false. ...It may be true, as Popper claims, that we cannot conclusively verify a hypothesis, but we cannot conclusively falsify it either. The most distinctive feature of computer simulation experiments is that the simulationist has complete control over the experimental conditions via (a) the random number streams driving the simulation model's stochastic input processes, and (b) the deterministic inputs governing model operation. Thus in simulated experimentation it is possible to isolate the effects of auxiliary assumptions, so that the Duhem-Quine problem can be effectively resolved. However as several colleagues have pointed out, often practitioners fail to evaluate the effects of auxiliary assumptions in large-scale simulation projects. This failure may be due to the lack of a well-documented, widely recognized methodology for addressing the Duhem-Quine problem in the context of simulation studies. Future simulation research should focus on the development of such methodology together with a comprehensive investigation of the connections between methods for solving the Duhem-Quine problem and methods for validating a simulation model. Beyond their theoretical objections to Popperian falsificationism, Woodward and Goodstein claim that this approach has serious practical disadvantages: Suppose a novel theory predicts some previously unobserved effect, and an experiment is undertaken to detect it. The experiment requires the construction of new instruments, perhaps operating at the very edge of what is technically possible, and the use of a novel experimental design, which will be infected with various unsuspected and difficult-to-detect sources of error. As historical studies have shown, in this kind of situation there will be a strong tendency on the part of many experimentalists to conclude that these problems have been overcome if and when the experiment produces results that the theory predicted. Such behavior certainly exhibits anti-Popperian dogmatism and theoretical "bias," but it may be the best way to discover a difficult-to-detect signal. Here again, it would be unwise to have codes of scientific conduct or systems of incentives that discourage such behavior. The scenario of Woodward and Goodstein is a remarkably accurate description of the experimental setting in which occurred all of the cases of pathological science detailed by Langmuir and Hall (1989) and Broad and Wade (1982). Moreover, this scenario describes the notorious cold fusion experiments of Martin Fleischmann and B. Stanley Pons as documented in the book Cold Fusion: The Scientific Fiasco of the Century by John R. Huizenga (1993). It seems clear that in such a scenario, the scientist's foremost concern should be to avoid lapsing into self-deception and pathological science. 4. THE SOCIAL STRUCTURE OF SCIENCE Woodward and Goodstein claim that ultimately inductivism and falsificationism are inadequate as theories of science because they fail to account for the psychology of individual scientists and the social structure of science. First Woodward and Goodstein consider the role of social interactions in scientific investigation: Suppose a scientist who has invested a great deal of time and effort in developing a theory is faced with a decision about whether to continue to hold onto it given some body of evidence. ... Suppose that our scientist has a rival who has invested time and resources in developing an alternative theory. If additional resources, credit and other rewards will flow to the winner, perhaps we can reasonably expect that the rival will act as a severe Popperian critic of the theory, and vice versa. As long as others in the community will perform this function, failure to behave like a good Popperian need not be regarded as a violation of some canon of method. Turning next to the psychology of individual scientists, Woodward and Goodstein explore the difficulty of sustaining the necessary long-term commitment of time and resources to a hypothesis without mentally exaggerating the supporting evidence and downplaying the contrary evidence--especially in the early stages of a project when belief in the hypothesis may be extremely fragile: All things considered, it is extremely hard for most people to adopt a consistently Popperian attitude toward their own ideas. Given these realistic observations about the psychology of scientists, an implicit code of conduct that encourages scientists to be a bit dogmatic and permits a certain measure of rhetorical exaggeration regarding the merits of their work, and that does not require an exhaustive discussion of its deficiencies, may be perfectly sensible. ... In fact part of the intellectual responsibility of a scientist is to provide the best possible case for important ideas, leaving it to others to publicize their defects and limitations. In contrast to this point of view, Peter Medawar, the winner of the 1960 Nobel Prize in medicine for his work on tissue transplantation, made the following statement in his book Advice to a Young Scientist (Medawar 1979, p. 39): I cannot give any scientist of any age better advice than this: the intensity of the conviction that a hypothesis is true has no bearing on whether it is true or not. The importance of the strength of our conviction is only to provide a proportionately strong incentive to find out if the hypothesis will stand up to critical evaluation. (The emphasis in the quoted statement is Medawar's.) Like Langmuir and Hall (1989), Medawar's Advice to a Young Scientist should be required reading for individuals at all stages in their scientific careers. Over the past twenty years, I have accumulated considerable experience in mediating extremely acrimonious disputes between researchers acting as "severe Popperian critics" of each other's work. Much of this hard-won experience was gained during the nine years that I served as a departmental editor and former departmental editor of the journal Management Science. To avoid reopening wounds which have not had much time to heal, I will not go into the particulars of any of these cases; but I feel compelled to draw some general conclusions based on these cases. In every one of the disputes that I mediated, the trouble started with extensive claims about the general applicability of some simulation-based methodology; and then failing to validate these claims independently, reviewers and other researchers proceeded to write up and disseminate their conclusions. This in turn generated a heated counterreaction, usually involving claims of technical incompetence or theft of ideas or both. Early in my career I served as the "special prosecutor" in several of these cases. Later on I moved up to become the "judge," and in the end I was often forced to play the role of the "jury" as well. In every one of these cases, ultimately the truth emerged (as it must, of course)--but the process of sorting things out involved the expenditure of massive amounts of time and energy on the part of many dedicated individuals in the simulation community, not to mention the numerous professional and personal relationships that were severely damaged along the way. In summary, I claim that when individual researchers violate Feynman's precepts of "utter honesty" and "leaning over backwards," the cost to the scientific enterprise of policing these individuals rapidly becomes exorbitant.
