So, we must be forced to decide between two competing hypotheses?
H0: Use classical hypothesis testing
H1: Use Bayesian analysis
Is there enough evidence to reject the null?
Or, what is the probability that the null is true if we reject it?
Milo Schield wrote:
> Dennis Roberts has issued a fundamental challenge: "if we had to opt one way
> OR the other (there is no middle groud) ... in our instruction related to
> statistics or analysis ... which way should we go: take a bayesian approach
> ... or, the way most have been doing it for seems like a zillion years?"
> ======================================
> Obviously the answer depends on one's goal(s). If our goal is to show
> students the deductive power of statistical inference, then the classical
> (non-Bayesian) answer is appropriate and most courses being taught today are
> in good shape. If our goal is to show students how to evaluate the results
> of classical statistical inference -- in the face of outside knowledge --
> then the Bayesian approach can be superior and most courses taught today are
> inadequate.
>
> I think our goal should be much broader. I think we should teach students
> how to use statistics as evidence. Given a statistical association, how can
> one evaluate the support it gives to non-statistical inferences? This
> question is fundamental to a lot of decision making. If this is the goal,
> then almost all courses being taught today are at least somewhat inadequate.
>
> On this basis, Bayesian thinking, although helpful, is not particularly
> fundamental or fruitful in dealing with the broader issues of what I call
> "statistical literacy: the use of statistics as evidence."
>
> These broader issues in "statistical literacy" include bias and confounding
> related to measurements, surrogate populations, lurking variables, etc.
>
> This broader view of statistical inquiry ties in with what Alan McLean said
> earlier:
> > It has been my opinion for quite some time now that the uncertainties in
> > conclusions due to use of surrogate populations, plus those due to
> > measurement (e.g. uncertainty in interpretation of questions in a
> questionnaire),
> > far exceed sampling errors - probably even exceed nonsampling errors.
>
> This broader view ties in with John Bailar's assertion that bias (including
> confounding) is typically/often more important than chance in interpreting
> the results of a statistical study. [Amstat News]
>
> This broader view ties in with Judea Pearl's essay ``Why There Is No
> Statistical Test For Confounding, Why Many Thing There Is, And Why They Are
> Almost Right''. Select "Other Papers" on his home page:
> http://singapore.cs.ucla.edu/jp_home.html [Not the underscore between jp
> and home.]
>
> This broader view ties in with several essays in Lynn Steen's "Why Numbers
> Count: Quantitative Literacy for Tomorrow's America" by The College Board
> (c.f., "Mere Literacy is Not Enough" by George Cobb, "Understanding the
> News" by Gina Kolata, and "Quantitative Literacy Across the Curriculum" by
> Glenda Price.) Cost $19.95 for paper back. http://www.collegeboard.org
>
> For more on Statistical Literacy, attend the ASA JSM 2000 session I
> organized on Statistical Literacy scheduled for Monday at 10:30 AM. Topics
> include: "Thinking Big about Statistics" by John Bailar, "Cross-Level
> Inference as an Identification Problem" by Phillips Shively (U.MN), "A
> Case Story in the Teaching of Observational Studies" by Chamont (Wei-hong)
> Wang, "A First-Year Interdisciplinary Quantitative Reasoning Course" by
> David Jabon and Carolyn Narasimhan, and "Teaching Statistics For Use in
> Epidemiology" by Joseph H. Abramson (author of Making Sense of Data).
>
> If I had to organize these matters into levels, I put it this way:
> 1st: Classical statistical inference
> 2nd: Bayesian reasoning [For details, see my post "Hyp test:better
> definition" (4/16/00)
> or my articles on Bayesian reasoning in teaching hypothesis tests and
> confidence intervals
> on my home page*
> 3rd: Reasoning about bias, confounding and systematic sources of variation.
> For more details, see my home page* for my article on "Statistical
> Literacy"
> 4th: Reasoning about causation. [See Robbins, Pearl, Shafer, etc.]
>
> The first level is pure deduction; the last three levels are mixtures of
> deduction and something more (induction, subjective priors, etc.).
>
> In summary, I agree with Dennis that we must move beyond classical
> (deductive) statistical inference.
>
> * Milo's home page: www.augsburg.edu/ppages/schield
>
> ====================================================
> Dennis roberts wrote in message
> <[EMAIL PROTECTED]>...
> >At 03:37 PM 4/18/00 -0400, Rich Ulrich wrote:
> <snip>
> >the problem i see ... stated in simple terms without lots of semantical
> >gobblygook ... is that we don't spend nearly enough time on thinking about
> >the questions we want to explore ... as researchers (ask joe ward) ... BUT,
> >we sure spend tons of time on learning inferential statistics ... so, the
> >bias in the field is the tendency to think that inferential statistics ...
> >and the logic behind it ... is THE way to knowledge ... but, it is not
> >(though it helps). see below
> >
> >in a way ... what we should do is to BAN ANY discussion of statistical
> >analysis ... UNtil we have a good grasp on the issue at hand ... or, if you
> >want to say it this way (if there is some deduction from some theoretical
> >position) ... formed a sensible hypothesis ... and if this takes time ...
> >or we have to revise it till we get something that is reasonable ... then
> >we need to take the time.
> >
> >then and ONLY then should we allow ourselves to ask: how can data analysis
> >help me in this quest to the answer to the questions i have posed ... or,
> >help me to sort out ways in which to test this deduction from the theory
> >that i have made ...
> >
> >so, to get this ball rolling along SOME line of inquiry ... let's pose the
> >basic question:
> >
> >if we had to opt one way OR the other (there is no middle groud) ... in our
> >instruction related to statistics or analysis ... which way should we go:
> >take a bayesian approach ... or, the way most have been doing it for seems
> >like a zillion years? (and so no one thinks i have loaded the deck ... i
> >don't really care which way we would good ... my only concern here is that
> >IF we have to make a decision ... how would we decide that?)
> >
> >this seems like a legitimate question to ask but, certainly, it would take
> >a lot of PRE data collection work (if it ever came to that point) ... to
> >focus in on subparts of this overall question ... and to try to define
> >important issues that would have to be dealt with ... before one could ever
> >be in a position to even conduct some 'study' about this .... and attempt
> >to arrive at some answer ...
> >
> >so, i offer a challenge: let's rationally discuss this question (not that
> >this is any better than many others that could be framed) ... and restrict
> >our discussion to NON statistical matters ... and see if we could develop a
> >plan that if implemented ... would help us answer the question of interest
> ...
> >
> >if we can do that ... THEN let's see what might be an appropriate way or
> >ways ... to handle any data that might come out of this exercise
>
> >===========================================================================
> >This list is open to everyone. Occasionally, less thoughtful
> >people send inappropriate messages. Please DO NOT COMPLAIN TO
> >THE POSTMASTER about these messages because the postmaster has no
> >way of controlling them, and excessive complaints will result in
> >termination of the list.
> >
> >For information about this list, including information about the
> >problem of inappropriate messages and information about how to
> >unsubscribe, please see the web page at
> >http://jse.stat.ncsu.edu/
> >===========================================================================
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
