Those who dislike long posts (20 kB), academic references,
cross-posting, or have no interest in "Comprehensive School Reform
Models" are urged to immediately hit the DELETE button.
ABSTRACT: I attempt to correct a misunderstanding by EvalTalk's Jacob
Silver of my 30 Nov 2005 post titled "Re: Study Rates 22 Widely Used
Comprehensive School Reform Models." Jacob wrote: "I may be missing
something, but how will the overly rigorous
randomized-control-trial-derived data rip the guts out of
inquiry-based science education?" I list four things that Jacob (and
perhaps other subscribers) may have missed.
In response to my post "Re: Study Rates 22 Widely Used Comprehensive
School Reform Models" [Hake (2005a)], Jacob Silver (2005) wrote in
his EvalTalk post of 1 Dec 2005 titled "Re: Study Rates 22 Widely
Used Comprehensive School Reform Models":
"I may be missing something, but how will the overly rigorous
randomized control trial derived data rip the guts out of inquiry
based science education?"
I evidently did not clearly explain my position. I did not mean to
state or imply that "overly rigorous randomized-control-trial-derived
data [will] rip the guts out of inquiry-based science education."
What I wrote was more nuanced:
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
In my opinion the AIR (2005) report is yet further reason to fear
that the U.S. Dept. of Education's (USDE's) direct instruction
juggernaut, FUELED IN PART BY AN **UNSCIENTIFIC** ALLEGIANCE TO
RANDOMIZED CONTROL TRIALS [RCT's] AS THE GOLD STANDARD OF EDUCATIONAL
RESEARCH . . . [see "a" below]. . . , will eviscerate effective K-12
guided inquiry-based science education in the U.S. [see e.g., Hake
(2004; 2005b,c,d)]. BTW, "guided inquiry-based" does NOT mean the
seldom used boogeyman "pure discovery learning" researched by the
widely misinterpreted Klahr & Nigam (2004).
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Among the things that Jacob may have missed are these four (a, b, c, d):
a. RCT's are NOT EVEN rigorous, let alone *overly* rigorous.
According to Cook & Payne: "In some quarters, particularly medical
ones, the randomized experiment . . .[commonly called randomized
control trial (RCT)]. . . is considered the causal 'gold standard.'
It is clearly not that in educational contexts, given the
difficulties with implementing and maintaining randomly created
groups, with the sometimes incomplete implementation of treatment
particulars, with the borrowing of some treatment particulars by
control group units, and with the limitations to external validity
that often follow from how the random assignment is achieved." For
further discussion of RCT problems see "The 2004 Claremont Debate:
Lipsey vs. Scriven" [Donaldson & and Christie (2005)] and Hake
(2005e).
b. According to my online Webster's Third New International
Dictionary, definition "1a" of "eviscerate" is "disembowel" (the
meaning assumed by Jacob Silver). But definition "1b" is
"devitalize," i.e., to deprive of life or vitality. IF the USDE were
threatening science education with a direct instruction (DI) dagger
then "disembowel" would be an appropriate meaning.
However, as indicated by the USDE's:
(1) financing of the *problematic* [see e.g., Kramer (2005), Hake
2005f)] AIR (2005) report,
(2) misinterpretation of Klahr & Nigam (2004) as generally supporting DI, and
(3) appointment of DI czar Douglas Carnine to the Technical Advisory
Group for the "What Works Clearinghouse,"
the USDE is riding DI as a juggernaut that devitalizes - OK - had I
been more metaphorically adept I would have written "the DI
juggernaut crushes."
c. My view is that it's the *USDE's DI juggernaut* that will
eviscerate "guided inquiry-based science education in the U.S.", NOT
randomized control trials per se.
d. The square bracketed reference "[see e.g., Hake (2004;
2005b,c,d)]", in the quote bracketed by "HHHHHHHHHHH. . ." above,
indicates that the explanation of why the DI juggernaut will
eviscerate "guided inquiry-based science education in the U.S." is
contained in Hake (2004; 2005b,c,d) [see REFERENCES below].
For example the abstract of "Will the No Child Left Behind Act
Promote Direct Instruction of Science?" [Hake (2005c)] reads:
"The No Child Left Behind (NCLB) Act requires testing in science
achievement starting in 2007. Will such testing tend to propagate
California's Direct Science Instruction (DSI) [Hake (2004)]
throughout the entire nation? After discussing the evidence for the
superiority of "interactive engagement" or "guided inquiry" methods
over DSI in conceptually difficult areas of science, I indicate seven
reasons why NCLB might promote DSI, and one reason - possible
*effective* intervention by the National Research Council. . .[see
e.g., Wilson & Bertenthal (2005)]. . . why it might not."
My seven reasons for why NCLB might promote Direct Science
Instruction are [see [Hake (2005c)] for references other than Hake
(2002, 2005e, 2005g), Heron & Meltzer (2005), Klahr and Nigam (2004),
Wieman & Perkins (2005)]:
1. Most interactive engagement and guided inquiry methods have not .
. .[usually for very good reasons (see, e.g. Hake 2005e)] . . . been
tested in randomized control trials (RCT's), the highly contested
"gold standard" of the U.S. Dept. of Education (USDE) and its "What
Works Clearinghouse" <http://www.w-w-c.org/>.
2. The heavily publicized [Adelson (2004), Begley (2004a,b),
Cavenaugh (2004a,b), Tweed (2004a,b), USDE (2004)] research of Klahr
and Nigam (2004) is widely misinterpreted as demonstrating the
general superiority of DSI.
3. It's easier to test for rote memorized material implanted by DSI than for
conceptual understanding of science and its methods induced by guided
inquiry or interactive engagement methods, as witness the inept
questions on California's STAR physics exam [Woolf (2005a,b,c), Hake
(2005g).
4. The U.S. Department of Education (USDE) is evidently bereft of
advisors from physical sciences, relying for advice on psychologists,
psychometricians, statisticians, economists, sociologists,
administrators, medical specialists, policy analysts, and education
specialists, most with a proclivity towards "Random Control Trials"
(RCT's).
5. Douglas Carnine (2000), dean of Direct Instructionists, is a
member of the Technical Advisory Group for the "What Works
Clearinghouse."
6. Psychologist Grover Whitehurst, director of the U.S. Education
Department's Institute of Education Sciences
<http://www.ed.gov/about/offices/list/ies/index.html?exp=0>),
evidently believes that "In science education, there is almost
nothing of proven efficacy" Begley [(2004b)]. . .[Whitehurst, as are
most psychologists, is evidently oblivious of the physics education
reform effort [Hake (2002), Heron & Meltzer (2005), Wieman & Perkins
(2005)]
7. Campbell's Law [Campbell (1975), Nichols & Berliner (2005)]: "The
more any quantitative social indicator is used for social decision
making, the more subject it will be to corruption pressures and the
more apt it will be to distort and corrupt the social processes it is
intended to monitor."
Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<[EMAIL PROTECTED]>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>
REFERENCES [Tiny URL's courtesy <http://tinyurl.com/create.php>]
AIR. 2005. "CSRQ Center Report on Elementary School Comprehensive
School Reform Models," November. A news release is at
<http://www.air.org/news/documents/Release200511csr.htm>; the
complete report may be downloaded at
<http://www.csrq.org/reports.asp>, or directly at
<http://www.air.org/news/documents/ES%20CSRQ%20Report%20-%20Full.pdf >(3.5
MB!); and the executive summary is at
<http://www.csrq.org/documents/ExecutiveSummary_001.pdf> (148kB).
Cook, T.D. & M.R. Payne. 2002. "Objecting to the Objections to Using
Random Assignment in Educational Research," in Mosteller & Boruch
(2002).
Donaldson, S.I. & and C.A. Christie. 2005. "The 2004 Claremont
Debate: Lipsey vs. Scriven: Determining Causality in Program
Evaluation and Applied Research: Should Experimental Evidence,"
Journal of MultiDisciplinary Evaluation, Part 1,
<http://evaluation.wmich.edu/jmde/JMDE_Num003.html>:
(a) 829 kB html at
<http://www.wmich.edu/evalctr/jmde/content/JMDE%20Num%203_files/Webpages%20JMDE%20003/JMDE_003_Part_I.htm>
or <http://tinyurl.com/dahjq>.
(b) 726 kB pdf at
<http://www.wmich.edu/evalctr/jmde/content/JMDE%20Num%203_files/PDFs%20JMDE%20003/JMDE_003_Part_I.pdf>
or <http://tinyurl.com/79n3b>.
Hake, R.R. 2002. "Lessons from the physics education reform effort,"
Ecology and Society 5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. Ecology and Society
(formerly Conservation Ecology) is a free online "peer-reviewed
journal of integrative science and fundamental policy research" with
about 11,000 subscribers in about 108 countries.
Hake, R.R. 2004. "Direct Science Instruction Suffers a Setback in
California - Or Does It?" AAPT Announcer 34(2): 177; online as
reference 33 at <http://www.physics.indiana.edu/~hake>, or download
directly by clicking on
<http://www.physics.indiana.edu/~hake/DirInstSetback-041104f.pdf> (420 KB)
[about 160 references and 180 hot-linked URL's]. A pdf version of the
slides shown at the meeting is also available at ref. 33 or can be
downloaded directly by clicking on
<http://www.physics.indiana.edu/~hake/AAPT-Slides.pdf> (132 kB).
Hake, R.R. 2005a. "Re: Study Rates 22 Widely Used Comprehensive
School Reform Models," online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0511&L=pod&O=D&P=21440>.
Post of 30 Nov 2005 14:09:43-0800 to AERA-C, AERA-D, AERA-B,
AERA-GSL, AERA-K, AERA-L, AP-Physics, ASSESS, IFETS, Math-Learn,
Phys-L, Physhare, PhysLrnR, POD,
STLHE-L, PPAS, PsychTeacher (rejected), TeachingEdPsych, & TIPS.
Hake, R.R. 2005b. "Will Evidence and Logic Reform Education? (was
California standards test in physics)," online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0502&L=phys-l&P=R1541>. Post
of 3 Feb 2005 11:29:26-0800 to AERA-D, AERA-H, AERA-K, AERA-L,
AP-Physics, ASSESS, Chemed-L, EvalTalk, Math-Learn, Phys-L, Physhare,
POD.
Hake, R.R. 2005c. "Will the No Child Left Behind Act Promote Direct
Instruction of Science?" Am. Phys. Soc. 50: 851 (2005); APS March
Meeting, Los Angles, CA. 21-25 March; online as ref. 36 at
<http://www.physics.indiana.edu/~hake>, or download directly by
clicking on
<http://www.physics.indiana.edu/~hake/WillNCLBPromoteDSI-3.pdf> (256
kB). See also Hake (2005d).
Hake, R.R. 2005d. "Seven Reasons Why The NCLB Might Promote Direct
Instruction of Science in the U.S. and One Reason Why It Might Not,"
online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0504&L=phys-l&F=&S=&P=1107>.
Post of 4 Apr 2005 15:03:45-0700 to AERA-C, AERA-D, AERA-G, AERA-H,
AERA-J, AERA-K, AERA-L, AP-Physics, ASSESS, Biopi-L, Chemed-L,
EvalTalk, Math-Learn, Phys-L, Physhare, POD, and STLHE-L.
Hake, R.R. 2005e. Re: Should Randomized Control Trials Be the Gold
Standard of Educational Research? online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0504&L=aera-l&T=0&O=D&P=2100>.
Post of 17/18 Apr 2005 to AERA-C, AERA-D, AERA-G, AERA-H, AERA-J,
AERA-K, AERA-L, AP-Physics, ASSESS, Biopi-L, Chemed-L, EvalTalk,
Math-Learn, Phys-L, PhysLrnR, Physhare, POD, STLHE-L, & TIPS.
Hake, R.R. 2005f. Re: Unit of Analysis and AIR study (was Re: Study
Rates . . .), AERA-D post of 2 Dec 2005 13:06:42-0800, online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0512&L=aera-d&T=0&F=&S=&X=37057D3B64CA7ABE2E&Y=rrhake%40earthlink.net&P=467>,
or <http://tinyurl.com/bxxhp>. I wrote: "It appears to me that
educational researchers generally should take a careful look at the
AIR report and publish their critiques in the open literature in an
attempt to derail USDE's direct-instruction juggernaut."
Hake, R.R. 2005g. "Re: WriteCDE (California Standards Test, Physics
2003/04)" online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0502&L=phys-l&P=R15655>. Post
of 18 Feb 200511:10:03-0800 to Phys-L, Phys-LrnR, Physhare, and
Math-Learn.
Heron, P.R.L. & D. Meltzer. 2005. "The future of physics education
research: Intellectual challenges and practical concerns," Amer. J.
Phys. 73(5): 390-394; online at
<http://www.physics.iastate.edu/per/articles/index.html>, scroll
down to "invited papers," or download directly by clicking on
<http://www.physics.iastate.edu/per/docs/Heron-Meltzer.pdf> (56kB).
Kramer, S. 2005. "Unit of Analysis and AIR study," AERA-D post of 1
Dec 2005 09:48:24-0500, online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0512&L=aera-d&T=0&F=&S=&X=4DB0F610532F1ADD3C&Y=rrhake%40earthlink.net&P=56>,
or <http://tinyurl.com/7grel>. Kramer wrote: "I think this [the
dangerous nonsense of reporting the Success for All longitudinal
study that used an appropriate unit-of-analysis as roughly equivalent
to a study of DI comparing 30 students in a single rural classroom
using DI to students in another classroom using a traditional basal
reader] is an issue that statisticians in AERA-D should address. Is
there a formal way to develop a position on this issue?"
Klahr, D, & M. Nigam. 2004. "The equivalence of learning paths in
early science instruction: effects of direct instruction and
discovery learning" (2004)
<http://www.psy.cmu.edu/faculty/klahr/papers.html>.
Mosteller, F. & R. Boruch, eds. 2002. "Evidence Matters: Randomized
Trials in Education Research". Brookings Institution.
Silver, J. 2005. "Re: Study Rates 22 Widely Used Comprehensive
School Reform Models." EvalTalk post of 1 Dec 2005 13:45:58-0500,
online at
<http://bama.ua.edu/cgi-bin/wa?A2=ind0512a&L=evaltalk&T=0&F=&S=&X=71590041D6753EC00A&Y=rrhake%40earthlink.net&P=553>
or
<http://tinyurl.com/dz7on>.
Wieman, C. & K. Perkins. 2005. "Transforming Physics Education,"
Physics Today 58(11):36-41, November; for a brief description see
<http://www.physicstoday.org/vol-58/iss-11/contents.html>.
Subscribers may download the full article at
<http://www.physicstoday.org/vol-58/iss-11/p36.shtml>.
Wilson, M.R. & M.W. Bertenthal, eds. 2005. "Systems for State Science
Assessment," Nat. Acad. Press; online at
<http://www.nap.edu/catalog.php?record_id=11312>.
---
You are currently subscribed to tips as: [email protected]
To unsubscribe send a blank email to [EMAIL PROTECTED]
