Re: What is an experiment?

[Brett A Magill]

Title: Re: What is an experiment?

Below is a fine description of experimentation is in perhaps the social or medical sciences.  But I think it excludes a lot of experimental work conducted in other fields. Isn't the defining element of experimentation really its counterfactual logic, that is in the absence of x, y would not have occurred.  The best way to establish this would be to administer some treatment and observe what happens while at the same time and place and with the same subject observing the outcome in the absence of that treatment in exactly the same conditions.  This would be an experiment, an experiment that is not possible, granted (save the ability to travel back in time--and maybe not even then).  Yet, this does not include randomization. Randomization is one way to *approximate* this ideal counterfactual.  Controlling for possible differences in time, place, and object.  However, some situations, for example experiments in chemistry or physics perhaps, a single observation of the effects of a treatment compared to conditions without that treatment might be sufficient to approximate this counterfactual logic due to the nature of the object of study. I suppose my point is that randomization is one way to apply counterfactual logic, but not the only way.  What's more, randomization is, though well accepted, only an approximation to an ideal.  Further, there may be other approximations that suffice for a given application.  Randomization is not a defining element of experimentation per se, though in many areas of application, essential to experimentation nonetheless. It is this counterfactual logic that is the defining element of experimentation.  This might open things up a bit.  A lot of observational work is done without experimental manipulation.  The question is really how good the approximation is to the ideal.  Observational studies in social sciences or biology do a poor job of this.  In the same area, a quasi-experiment does better, and randomized experiments are a pretty darn good approximation.  Is manipulation by the experimenter a required element?  As I think about this, I am not sure.  Experimental manipulations are certainly one way to approximate the counterfactual, and in some situations absolutely necessary.  But again, depending on the application, it might not be necessary to get a good approximation of the ideal.  Naturally existing conditions that create a "treatment" and "non-treatment" condition might be sufficient.  Why not?  If conditions in absence of any intervention on the part of the "experimenter" create a counterfactual condition and the objects of observation are stable enough, why can causation not be attributed.  Again, it requires some faith in the nature of the object under study and its stability. Anyway, just some musings, and I hope it is interpreted that way.  I am interested in hearing the reaction of others (unless they are flames--criticism I can take).   -----Original Message----- From: Paul Bernhardt [mailto:[EMAIL PROTECTED] Sent: Wed 3/12/2003 6:19 PM To: Edstat Cc: Subject: Re: What is an experiment? User968758 said on 3/12/03 2:01 PM: >One definition of an experiment might be that it is a plan for gathering >data, >designed to give informative answers to the questions of interest. How one >"models" the data to get at these answers differs depending on the >experiment. >One might use a regression model, a survival analysis model, a time series >model, etc.... > Most generally, we may describe a 'study' as any systematic gathering of data for the purposes of gaining an accurate understanding of a phenomena. An Experiment is simply one type of study. Experiments are very precisely defined and have the following characteristics. 1) Two or more conditions in the study, one of which is considered a 'control' condition. 2) A variable, manipulated by the experimenter, which distinguishes the conditions described above. 3) Subjects in the study are randomly assigned to conditions in the study. That is, each subject which enters the subject has an equal chance of assignment to each condition and one subject's assignment has no influence on another subject's assignment. 4) At least one dependent variable is measured. When some of the above conditions are relaxed, you may still be doing a scientific study, but not necessarily an experiment. For instance, sometimes the groups/conditions are pre-existing and random assignment of individuals to the groups is not possible and these are often called Quasi-Experiments. A classic example of this would be Sex with levels Male and Female. One may still do a good scientific study to distinguish differences between the two groups. One may also recognize that conditions as distinguishing groups may not have meaning, but we may still do a correlational study by measuring more than two variables on a randomly obtained sample. In such a case a relationship between the two variables may be explicated even though no experiment has been conducted. Hope that all helps. Paul