Yes, the ANOVA application is a good example - I'm happy with that.
With regard to the confidence interval, what you seem to be describing is what I usually point out to students after dealing with confidence intervals and in introducing hypothesis testing - that you can use a confidence interval to decide that there is a 'significant difference' between the sample statistic and the hypothesised value of the parameter. You are to my mind emphasising that you can consider multiple, or to some extent undefined, hypothesised values. In this sense I agree with you. All the same, if you are using a critical value approach, the critical values define a significant distance from the parameter. This is the whole basis of formal hypothesis testing.
Regards,
Alan
On Tuesday, November 19, 2002, at 04:44 PM, Donald Burrill wrote:
On Mon, 18 Nov 2002, Alan McLean wrote in part:Hi Don,Thanks for your reply - as usual, thorough, with lots to think about.AM> If Z = (X - mu)/sigma is ~ N(0, 1), then is AM> T = (X - mu)/s (where s is the sample SD based on a simple random AM> sample of size n) ~ t(n-1)?DB> Short answer: Yes.
DB> Longer answer: the number of degrees of freedom for the t
DB> distribution for such a statistic is the number of degrees of
DB> freedom associated with the variance estimate (well, with its square
DB> root) in the denominator.
I believe not. After all, mean and variance are mutually independentThis is certainly the case. However, my uncertainty lies in that the sample on which the variance estimate s^2 is based is not in itself linked to the value of X. A related question is whether or not in the usual application T = (Xbar - mu)/[s/sqrt(n)] the sample SD used, s, has to refer to the same sample as Xbar?
quantities in normal (aka Gaussian) distributions, and generally in
empirical data. But consider ANOVA, where all the observations in the
experiment are used in estimating a pooled error variance (aka error
mean square, MS(E)); and in either post hoc contrasts or planned
comparisons, which characteristically involve only a proper subset of
the available estimates of means, MS(E) characteristically appears in
the denominator of a quantity that is, essentially, a t-ratio (although
it is not usually referred to the t-distribution, but to the square root
of critical values from a certain F-distribution). So in such cases,
the sample on which the SD is based is not identical to the samples on
which the means are based.
AM> My second question is on the matter of confidence intervals. <snip>
<large snip>
AM> So my question is: how do YOU explain to students what a confidence
AM> interval REALLY is?
DB> A C.I. is an observed instance of a random variable, representing DB> the range of values that one might specify in a null hypothesis and DB> NOT have the hypothesis rejected.How can this be? The acceptance region of the test is based on the hypothesised value, while the confidence interval is based on the observed value.Yes, precisely. As I remarked in the next paragraph.OK, I'll try to restate the matter in somewhat different terms.This is I think another way of expressing the root of my uncertainty.
In hypothesis testing (HT) one takes the following point of view
(for convenience, I write about an average or mean, but the argument
is much the same for some other parameter of interest):
For the variable of interest that we are observing (X), there exists a
population mean value, mu. We don't know what its value IS, but we may
have some theory that tells us that the value ought to be, say, mu_0 (at
least, in the best of all possible worlds, or in the circumstances under
which we have gone out into the wilderness to look). Given such a
value, we ask whether the data we have observed be consistent with it.
To deal with this question we reduce "the data we have observed" to a
single number (say, a sample mean), acknowledge that this number is a
random variable (since its value depends on which particular cases we
happen to have observed), and make some assumptions about the nature of
the distribution of that random variable. [Details omitted here for
brevity!] Armed with this information, and the distribution being so
conveniently one we think we know something about, we can refine the
question "whether the data be consistent with mu_0" to the form
"Would these data [i.e., this sample mean] be unlikely to have been
observed, IF the true value of mu were indeed mu_0 (and if our
distributional assumptions are true, or at least tenable)?"
If our analysis leads us to believe that the conditional probability of
(observing a value as distant from mu_0 as the present data are) is less
than a criterion value (commonly symbolized by alpha and called 'level
of significance'), we decide to reject the hypothesis that the true
value of mu is mu_0. Having rejected that hypothesis, we are forced
(by the logic of elimination) to accept an alternative hypothesis,
which in most cases is the logical complement of the hypothesis we have
tested (commonly called the 'null hypothesis').
We often present a diagram somewhat prettier than this one, in that it
usually displays the pleasing curves of a normal distribution. But the
point of the display is to show a number line containing mu_0, the
hypothesized parameter value, and the critical values that bound the
regions where a sample mean would lead to rejection of the null
hypothesis and the region where a sample mean would NOT lead to
rejection. Thus:
---------+-------------------+-------------------+----------
L mu_0 U
where L is a lower boundary (for a specified alpha): if the sample
mean falls to the left of L (if xbar < L) the null hypothesis H_0 is
rejected; U is a corresponding upper boundary (if xbar > U, B_0 is
rejected); and mu_0 is at the location shown. The interval or region
between L and U is called the 'acceptance region', and if xbar falls in
this region we cannot reject H_0. The region (divided into two parts)
outside that interval -- that is, values less than L and values larger
than U) is called the 'rejection region'.
("Acceptance region" is arguably a misnomer, since one has no business
accepting the null hypothesis in the absence of a power analysis, which
would enable one to state a (conditional) probability of being wrong in
acceptance, aka the probability of a "type 2 error". However...)
It is customary, but not necessary, for the acceptance region to be
symmetrical about mu_0. This is convenient because for a symmetric
distribution (e.g., normal) a symmetric interval is the shortest
interval that meets the criterion implied above: that if H_0 is true
(that is, if mu=mu_0), the probability of a sample mean falling between
L and U is (1 - alpha). But there is no otherwise compelling reason why
one could not use asymmetric boundaries.
If the data are NOT so unlikely as all that (i.e., if that conditional
probability, commonly called the 'p-value', is larger than alpha), we
have not logical basis for rejecting the null hypothesis.
Now, we started out above by assuming that there was a magic value mu_0
that might, imaginably, be the true value of mu; and we thought (though
I hadn't said so earlier) that there was some usefulness, or meaning,
attached to the decision whether the data were consistent with the
proposition that mu = mu_0 (that is, with the null hypothesis).
[I can supply examples of values that might be interesting to test, in
various fields of enquiry; but so, I expect, can you, so I won't take
the time & space here.]
But suppose we don't know enough about the field under investigation to
be able to specify an interesting or useful possible value of mu?
The machinery that we have set up, to ask whether the observed sample
mean is far enough from a specified parameter value to permit us to
reject the hypothesis specifying that parameter value, can serve equally
well in the other direction. Here's that number-line again, but now I
do not specify a value for mu. Suppose the sample mean falls exactly at
L. Then for whatever value of mu this would be the case, I cannot
reject an hypothesis that speficies that value; and a fortiori I cannot
reject any hypotheses specifying values between U (= xbar) and mu. So
this value of mu (call it mu_max) is the largest value that is
consistent with the observed sample mean xbar. (Here by "consistent
with" I mean that a null hypothesis specifying a value > mu_max would
be rejected, but any null hypothesis specifiying mu such that
xbar < mu < mu_max would not be rejected.
---------+-------------------+-------------------+----------
L mu U
xbar mu_max
OTOH, suppose the sample mean xbar falls exactly at U:
---------+-------------------+-------------------+----------
L mu U
mu_min xbar
Then this value of mu (mu_min) is the smallest possible value which, if
tested by a HT, would not reject the hypothesis. Values of mu < mu_min
would lead to rejection of the hypotheses specifying them.
The interval between mu_min and mu_max is the (1 - alpha) confidence
interval about xbar. It is of the same length as the acceptance region
about mu_0 above; but it sense is reversed, in that the left end of the
C.I. corresponds to the right end of the acceptance region, and vice
versa. (This point is hardly ever understood, because the ideas of HT
and of CIs are commonly illustrated with symmetric sampling
distributions AND with symmetric upper and lower boundaries. I've often
thought that a proper treatment of the subject would include a few
instances, at least for illustration, of taking alpha = .05 (say) and
assigning .04 of that to the left end of the acceptance region and .01
to the right end. It would be straightforward to show (a) that this
acceptance region is longer than the one defined by assigning .025 to
each end and (b) that the corresponding CI is also asymmetrical, but in
the other direction.)
Well, that's about all I have time for this night. Perhaps it will
stimulate some further questions. -- Don.
-----------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
56 Sebbins Pond Drive, Bedford, NH 03110 (603) 626-0816
[was: 184 Nashua Road, Bedford, NH 03110 (603) 471-7128]
. . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
