(This post is formatted for viewing with a fixed-width font, such
as Courier.)
In a 99/5/9 post I suggest that relationships between variables
are more interesting for students than univariate distributions.
Referring to this point, Ronan Conroy writes (on 99/5/10)
> This is a fundamental point. Ampere, in a crucial development
> in the philosophy of science, pointed out that science does not
> study things in themselves, but the relationships between their
> attributes.
The "things" Ronan refers to are what I call "entities". The
"attributes" are (essentially) what I call "properties" or "vari-
ables".
(I discuss why I recommend the word "entity" over other candidate
words in a paper [1998, app. E.1]. I recommend the word "prop-
erty" over the word "attribute" because word frequency statistics
suggest that beginners are more familiar with the former word
[Breland and Jenkins 1997]. I discuss my view of the relation-
ship between the words "property" and "variable" in a paper
[1998, app. E.2].)
I fully agree with Ronan's point that science (together with all
the other branches of empirical research) generally studies rela-
tionships between properties (or relationships between attributes
or relationships between variables). I discuss this idea further
in a paper (1999, sec. 3.4).
André-Marie Ampère (Andre-Marie Ampere, 1775-1836) emphasized the
concept of a relationship between a certain type of entities --
entities he called "phenomena" (Williams 1970, 1989; Hofmann
1996). Since both variables and properties can be conceptually
viewed as "phenomena", Ampère was clearly close to the concept of
'relationship between variables'.
Probably Ampère viewed "phenomena" as being equivalent to what
modern speakers of English refer to as "variables". However, I
have been unable to confirm this because Ampère wrote in French,
and my French is weak, and I have not found any English transla-
tions of Ampère's detailed philosophical writing. So I have been
unable to directly study his philosophical writing to see how he
characterized his concept of 'phenomena'. Do any readers have
further information about Ampère's characterization of phenomena
or about translations of his philosophical writing?
* * *
Ampère brilliantly recognized and described the scientific method
(Williams 1989). The method (which is also called the
hypothetico-deductive method of science) consists of the follow-
ing four steps:
1. A researcher frames (i.e., invents) a new hypothesis about
some area of experience. (The researcher frames the hypothe-
sis on the basis of knowledge of earlier research, intuition,
and logic.)
2. A researcher (perhaps the same researcher) then deduces or in-
fers an empirically testable implication of the hypothesis.
3. A researcher (perhaps the same) then performs an empirical re-
search project to test whether the implication is actually
present in the area of experience.
4. If evidence of the implication is found (and in the absence of
a reasonable alternative explanation), the community with
prime interest in the area of experience will (by informal
consensus) accept (or will be more inclined to accept) the hy-
pothesis framed in step 1 as being correct.
Ampère's description of the scientific method is important be-
cause most modern scientific research (and most other empirical
research) proceeds formally according to this method.
(I discuss the formal and informal aspects of the progress of
science in appendix A.)
Examination of instances of the use of the scientific method sug-
gests that the implication in step 2 can usually be usefully
viewed as a statement of a relationship between variables in some
population of entities. Thus the key statistical concept of 're-
lationship between variables' plays a central role in the scien-
tific method.
* * *
Ampère's empirical interests were in part in the branch of phys-
ics known as electrodynamics, which he founded (Williams 1989).
Perhaps his most important contribution in electrodynamics is the
discovery of the following relationship between variables:
The magnetic force between small elements of two cur-
rent-carrying conductors is inversely proportional to
the square of the distance between them and is directly
proportional to the product of the currents. (Ampère
also described how the three angles that define the
relative directions of the two conductors affect the
size of the force.)
The force described in this relationship is important because it
is the force that drives almost all electric motors. Scientists
have recognized the importance of Ampère's work in electrodynam-
ics by naming the unit of electric current, the ampere, in his
honor.
* * *
Ampère's contributions lead to questions about the history of re-
lationships between variables. Researchers have been formally
studying such relationships for centuries (with differing degrees
of emphasis in different eras). For example, the Pythagorean
theorem describes a formal relationship between variables that
was discovered by Pythagoras in the 6th century BCE.
Going back further still, humans (without being aware of it) have
been informally using relationships between "variables" since
when we began to reason. The aphorisms "haste makes waste" and
"power corrupts" are good examples of informal statements of re-
lationships between variables.
The use of relationships between variables is not limited to hu-
mans and may also exist in animals. That is, we could build
plausible models of some animal reasoning in terms of an animal
(unconsciously) learning about simple relationships between
"variables" that reflect properties of its habitat. The animal
uses the knowledge of the relationships to aid in its survival.
For example, an animal might learn a relationship between the
time of day and the likely location of prey. Such a learning ap-
proach to survival enables an animal to adjust as quickly as pos-
sible (sometimes within seconds or minutes) to unpredictable
changes in its habitat, such as changes due to a fire, a flood,
or an epidemic among its prey. These changes invariably cause
new relationships between variables to arise. It is in an ani-
mal's survival interest to learn as quickly as possible about all
relationships that have response variables that affect its sur-
vival or reproductive capability.
Viewing learning as learning about relationships between vari-
ables is also reflected in the literature of machine learning.
For example, Vapnik, in the first sentence in the first chapter
of his book THE NATURE OF STATISTICAL LEARNING THEORY, writes
In this book we consider the learning problem as a prob-
lem of finding a desired dependence using a limited num-
ber of observations (2000).
Vapnik goes on to show that the "dependence" he refers to is a
dependence (i.e., a relationship) between a single response vari-
able and zero or more predictor variables.
* * *
Ampère studied relationships between variables in the field of
physics which (together with chemistry, astronomy, and engineer-
ing) is one of the physical sciences. As I discuss in a paper,
most empirical research in the physical sciences can be usefully
viewed as studying relationships between variables (1999, app.
B).
When physical scientists study relationships, the standard sta-
tistical concepts of 'population' and 'sample' are often fuzzy or
not present. This is because in the physical sciences any "sen-
sible" sample of entities will often perform quite well as a rea-
sonable sample from the (implicit) population.
For example, Isaac Newton's law of cooling states that the rate
of cooling of a physical object when it is hotter than its envi-
ronment is proportional to its excess temperature. The model
equation for this relationship between variables is
r = k * deltaT
where
r is the rate of cooling (in degrees per unit of time)
deltaT is the excess temperature of the physical object above
the ambient temperature and
k is a constant. (k is constant within any given physical
object, but varies from one physical object to the
next.)
Newton derived this relationship from empirical research in which
he measured the excess temperature and computed the rate of cool-
ing at several times in a sample of relatively hot physical ob-
jects. In all the cases in the sample he found strong evidence
that this specific relationship between variables was present.
Thus Newton generalized the findings to his law. Because the law
has been repeatedly verified, physicists believe it applies to
all macroscopic physical objects in the population of physical
objects at all times.
(It is possible [and may be known] that the constant k in
Newton's law of cooling is not a perfect constant. Instead, k
may be better viewed as varying slightly as a function of tem-
perature, excess temperature, or possibly other variables. All
relationships between variables may be refined as measurement
technology improves.)
Newton's discovery of the law of cooling illustrates how in em-
pirical research in the physical sciences we may not need to pay
much attention to the population and sample -- the relationship
between variables under study is exactly the same throughout a
very broad population. On the other hand, in most other areas of
empirical research (including the biological and social sciences)
the definitions of the population and sample are important be-
cause in these areas a relationship between variables may change
substantially if we change to a slightly different population or
sample.
* * *
If we use the instruments and procedures Newton used to investi-
gate the law of cooling, we find that the error variation in the
relationship in research data reflecting the law is negligible.
Thus when Newton discovered the relationship he did not need to
use statistical methods to study the relationship. Instead, he
could see strong support for his conclusion that the relationship
reflected in the law is present by merely scanning his research
data, or by a few simple mathematical operations on the data, or
by plotting the data on scatterplots.
More generally, because of the (usually) low error variation in
relationships between variables studied in the physical sciences,
physical scientists often do not use statistical methods beyond
standard graphics to study relationships between variables. (To
fit model equations they sometimes use homegrown algebra and
sometimes statistical methods.) On the other hand, in other ar-
eas of empirical research the error variation is usually large
enough that the use of statistical methods greatly simplifies the
study of relationships between variables. In addition, statisti-
cal methods can reliably detect subtle phenomena in data that may
otherwise go unnoticed.
* * *
Researchers in the physical sciences may have difficulty if they
omit using statistical methods -- witness the cold fusion contro-
versy. The key issue in this controversy can be usefully viewed
as whether certain unexpected relationships between variables are
present in a cold fusion cell.
A cold fusion cell is an electrolytic cell with a palladium cath-
ode and often with a platinum anode and containing a solution of
deuterium (heavy water) and lithium deuteroxide. The original
cold fusion researchers reported that, under certain conditions,
if a cold fusion cell was connected to a source of electric cur-
rent, the heat energy output (response variable) was substan-
tially greater than the electrical energy input (predictor vari-
able). Such a relationship between variables contradicts ac-
cepted physical theory. The original researchers believed that
the (perceived) unexpected extra energy output of the cell was
due to the nuclear reaction of fusion taking place in the cathode
-- nuclear fusion gives off heat. Were it true that nuclear fu-
sion was occurring in these cells, it would have been extremely
important because it would suggest that we could use cold fusion
as a new safe and inexpensive source of energy.
Some researchers reported that (using their own cold fusion
cells) they were able to replicate the relationship between vari-
ables reported by the original researchers. But many other re-
searchers reported that they were unable to replicate the rela-
tionship (and were unable to demonstrate other implied relation-
ships between variables). In the end, the weight of evidence led
most physical scientists to conclude that the claimed new phenom-
ena (which can be usefully viewed as relationships between vari-
ables) are not present (Huizenga 1993).
The cold fusion controversy lasted several years and consumed a
substantial amount of physical scientists' resources. As sug-
gested by the government panel that investigated cold fusion, the
high resource consumption may have been partly because some re-
searchers involved in the controversy were not using proper sta-
tistical methods to determine whether the claimed new relation-
ships between variables were present (Energy Research Advisory
Board 1989, app. 2.C).
If the original cold fusion researchers had used proper statisti-
cal methods for detecting relationships (i.e., statistical tests,
a proper taking account of negative results, and experimental de-
sign considerations), it seems likely that the repeated high or
borderline p-values would have quickly alerted them to the fact
that that the sought-after relationships between the variables
were probably not present. Or if the relationships WERE present,
they were not properly detectable with the current experimental
methods, and thus further "more powerful" research must be per-
formed before a responsible positive conclusion could be drawn.
This might have saved the original researchers considerable em-
barrassment and might have saved the physical science community
substantial costs.
Omission of the use of statistical methods is not the only possi-
ble cause of the cold fusion errors. As also discussed by the
Energy Research Advisory Board panel, some of the mistaken posi-
tive conclusions may have arisen through errors that were made in
the measurement methods the original researchers used -- errors
that caused the measured net energy output of some cold fusion
cells to be over-estimated (1989, sec. II, app. 2.C).
Cold fusion research is associated with the sub-domain of chemis-
try called "electrochemistry". But regardless of whether the do-
main is chemistry, or medicine, or psychology, or any other area
of empirical research, a researcher can marry statistical methods
with proper measurement methods in the domain. This marriage,
when tempered with the proper cautions, yields optimal empirical
procedures to detect and study relationships between variables.
> An attribute is the relationship between a measurement process
> and the thing to be measured,
Ronan gives a definition of the concept of 'attribute' or 'prop-
erty'. He defines the concept in terms of three other concepts,
namely, 'relationship', 'measurement process', and 'entity'
(i.e., "thing to be measured").
Ronan uses the concept of 'measurement process' to define the
concept of 'property'. Thus he appears to view the concept of
'measurement process' as being more fundamental than the concept
of 'property'. Ronan's view does not seem to be contradictory,
but it is different from mine. I view the concepts of 'entity'
and 'property' as being verbally UNDEFINED, just like the con-
cepts of 'mass', 'length', 'time', and 'temperature' are usually
verbally undefined in physics.
(Mass, length, time, and temperature are all properties of enti-
ties. Mass and length are properties of physical objects. Time
[duration or point in time] can be viewed as a property of events
or a property of "reality" [the entity that contains all other
entities]. Temperature can be viewed as a property of environ-
ments. The ways of MEASURING these properties are clearly de-
fined in physics, but the concepts themselves are verbally unde-
fined. I discuss some possible objections to these points in ap-
pendix B.)
The concepts of 'mass', 'length', 'time', and 'temperature' are
verbally undefined in physics because it is not possible to ver-
bally define every concept in a field without introducing unde-
sirable circularity. Therefore, physicists have been forced to
choose certain of their concepts to be verbally undefined. It
makes sense to choose the simplest concepts in a field to be ver-
bally undefined and then, when possible, to define the more com-
plicated concepts in terms of the simpler undefined concepts.
Hence the concepts of 'mass', 'length', 'time', and 'temperature'
are verbally undefined in physics.
Dictionaries generally give definitions of the four undefined
concepts because such definitions provide the most benefit for
dictionary users. However, examination of the definitions and
examination of the definitions of the terms used in the definien-
tia invariably reveals (of necessity) that the set of definitions
associated with any of the four concepts is circular. Thus with-
out outside help we cannot use these definitions to explain or
understand the four concepts. Thus although the four concepts
have dictionary definitions, they are (due to the circularity of
the definitions) still effectively undefined.
If no (non-circular) verbal definition is available for a par-
ticular concept, how can we understand it? We can understand a
verbally undefined concept through "ostensive" definitions of it.
An ostensive definition is one that does not rely on words. In-
stead, the definition works through the defining person physi-
cally pointing at the thing being defined, or it may work through
some other form of direct experience. The direct experience al-
lows one to learn the concept without having to use words (except
for the word or words naming the concept being defined).
For example, although the fundamental properties of 'mass',
'length', 'time', and 'temperature' have no verbal definitions in
physics, they have ostensive definitions. These definitions are
not given in words and thus do not appear in formal discussion.
(The definitions are generally taken for granted in formal dis-
cussion.) Instead, the four definitions are given through direct
experience -- experience that occurs many times for most people
before they are eight years old.
For example, consider the property of the mass of a physical ob-
ject. (As one learns in high-school physics, this property is
basically the same as the property of the weight of a physical
object. The difference between these concepts, which is some-
times important, is not relevant here.) We can teach the prop-
erty of mass to an interested child who does not understand it by
handing them various physical objects and saying "heavy", "very
heavy", "light", and so on, as appropriate. Children can quickly
learn any perceivable property through such ostensive definitions
if enough opportunities are available for them to properly dis-
criminate different values of the property directly.
As infants we learn our first words (e.g., "mommy", "TV",
"green") through informal ostensive definitions that occur in the
speech and actions of the people around us. Furthermore, most
people learn almost all the words they use in their day-to-day
conversation through ostensive definitions, not through verbal
definitions. Thus ostensive definitions are more basic than ver-
bal definitions.
In an approach that is identical to the approach taken in phys-
ics, I view the concepts of 'entity' and 'property' as verbally
undefined fundamental concepts in empirical research (and also
verbally undefined fundamental concepts in broader day-to-day hu-
man reality). Humans understand these concepts through the many
ostensive definitions of them we experience and because the con-
cepts are the basis for a substantial part of our thinking.
I further discuss the role of the concepts in thinking in the
1999 paper (sec. 3).
* * *
Ronan's definition introduces the important concept of a "meas-
urement process" -- a process that we use to measure the values
of a property. For any given property of entities, researchers
may view the property as if there is one specific property, but
there may be several ways of measuring the value of the property
in entities.
For example, we can measure a person's weight (or mass) using a
standard bathroom scale (which may be a spring balance), or we
can use a pan balance in which we balance the person's weight
against known weights, or we can use a weight guesser from a car-
nival. All these methods are valid methods for measuring a per-
son's weight in the sense that they yield estimates that are
highly correlated with each other. However, these measures are
of different quality in the sense that they have different preci-
sion (i.e., high or low random variation about their expected
value) and different bias (i.e., expected deviation from the
"true" value).
* * *
The preceding discussion leads to the philosophical question of
what it means to speak of the "true" value (at a given time) of a
property of an entity. This question breaks into two cases: the
case in which a property is represented by a discrete variable
and the case in which a property is represented by a continuous
variable.
With a discrete variable we can certainly speak of and sometimes
even know the "true" value of the underlying property. For exam-
ple, if we wish to know the number of multiple-choice questions a
student answered correctly on a test, we can, if we are careful
enough, know the "true" value of this number (which can be use-
fully viewed as a property of the student).
In the more interesting case of a continuous variable, empirical
researchers usually define the "true" value of a property in
terms of some commonly-agreed-upon measurement approach because
this facilitates communication and understanding. For example,
researchers in the physical sciences usually define the "true"
values of the properties they study in terms of the definitions
and standards provided by the International Bureau of Weights and
Measures.
To illustrate measurement standards, suppose we decide to measure
the mass of physical objects using the kilogram as the unit of
mass. The exact formal English version of the definition of the
kilogram in the International System of Units (SI) is as follows:
The kilogram is the unit of mass; it is equal to the mass
of the international prototype of the kilogram (BIPM
2001a).
The international prototype of the kilogram is a platinum-iridium
cylinder kept under carefully controlled conditions in a labora-
tory of the International Bureau of Weights and Measures in
Sèvres, France. The national standards organizations of most
countries of the world accept the definition that this cylinder
has a mass of exactly one kilogram.
Using an accurate balance it is possible to make copies the in-
ternational prototype of the kilogram so that the copies have
masses that are very close to the mass of the prototype. Several
such copies and many copies of these copies have been made. From
these copies it is possible to create other physical objects that
have masses that are known multiples of one kilogram or that are
known fractions of one kilogram.
After we have obtained appropriate accurate masses we can use
them to calibrate other very accurate mass-measuring instruments.
(Such instruments, when properly calibrated, are more efficient
than using a pan balance with weights.) The "true" mass in kilo-
grams of any physical object is the mass we would get if we were
to measure the object with a calibrated mass-measuring instrument
AND
1. we had made no errors (no matter how small) in making our sec-
ondary masses from the international prototype of the kilogram
AND
2. we had made no errors (no matter how small) in calibrating the
instrument with the secondary masses AND
3. the instrument perfectly interpolates between or extrapolates
away from it calibration point(s) AND
4. the instrument has precision (at least) to the level of the
mass of sub-atomic particles.
Clearly, none of the four requirements can be satisfied. Thus
even with mass -- a fundamental concept of physics -- we cannot
determine the true value of this property for a given physical
object to perfect precision. Nor does it seem likely that humans
ever will determine the "true" mass of an arbitrary physical ob-
ject or the true value of any other continuous property.
However, if we are careful with our measurement procedures, we
can ESTIMATE the true value of any property to a very high preci-
sion, which is usually all we need. For example, currently the
best mass-measuring instruments for small objects can accurately
resolve ten or more significant digits of mass (BIPM 2001b). It
is hard to imagine situations in which we would need more accu-
racy than that.
Although we can generally never know the true value of a continu-
ous property of an entity, defining such "true" values is a use-
ful idealization because it encourages us to strive to accurately
measure the values. This leads to more accurate knowledge of re-
lationships between variables, which leads to better ability to
predict and control.
-------------------------------------------------------
Donald B. Macnaughton MatStat Research Consulting Inc
[EMAIL PROTECTED] Toronto, Canada
-------------------------------------------------------
APPENDICES AND REFERENCES
Because the appendices to this post are of less general interest,
I give only their titles here:
Appendix A: Formality And Informality In The Progress of Science
- this appendix is accessible at
http://www.matstat.com/teach/p0044.htm#a
Appendix B: Further Discussion Of Undefined Concepts In Physics
- this appendix is accessible at
http://www.matstat.com/teach/p0044.htm#b
The full post, with the appendices and references, is at
http://www.matstat.com/teach/p0044.htm
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
