Hake, R.R. 2003. "Re: Normalized Gain (was Inquiry method and motivation)," post of 24 Nov 2003 17:12:05-0800 to EvalTalk, Math-Learn, PhysLrnR, & POD; online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0311&L=pod&O=D&P=18573>. Later sent to AERA-D, ASSESS, and Biopi-L; and in abstract form to Chemed-L, Phys-L, Physhare, STLHE-L, edstat, FYA, and AP-Physics.
If you're:
(a) not interested, please hit the delete button.
(b) interested in scanning the entire post, click on the above URL
(c) interested in scanning an abstract see the APPENDIX.
Richard Hake, Emeritus Professor of Physics, Indiana University 24245 Hatteras Street, Woodland Hills, CA 91367 <[EMAIL PROTECTED]> <http://www.physics.indiana.edu/~hake> <http://www.physics.indiana.edu/~sdi>
AKPPENDIX [Abstract of Hake (2003)]
In his Math-Learn post of 21 Nov 2003 titled "Re: Normalized Gain." <[EMAIL PROTECTED]> asked three good questions:
1. Can you tell me more about normalized gain, and/or suggest a good resource? . . . .
2. Is "normalized gain" valid on a non-standardized test?
3. Can I compare "normalized gain" even though I changed tests, a couple of years ago, because we up-dated the text book? . . . . or should I limit my comparisons to before the change and after the change?"
In the following post I attempt to answer the above three questions and also consider two others:
4. Are Clement's comments on the "average normalized gain" <g> and the effect size "d" correct?
5. Is pre/post testing becoming more popular in disciplines other than physics?
I. DEFINITION OF THE AVERAGE NORMALIZED GAIN <g> In Hake (2002b) I wrote (slightly edited):
"The half-century old average normalized gain <g> for a treatment has been independently defined [Hovland et al. (1949), Gery (1972), Hake (1998a)] as <g> = Gain/[Gain (maximum possible). In terms of %scores
<g> = (<%post> - <%pre>) / (100% - <%pre>)
Where angle brackets <. . . .> indicate the class average (preferably only for students who have taken both the pre and post tests).
Thus, e.g., if a class averaged 40% on the pretest, and 60% on the posttest then the class-average normalized gain <g> = (60% - 40%)/(100% - 40%) = 20%/60% = 0.33. Ever since the work of Hovland et al. (1949) it's been known by pre/post cognoscente (up until about 1998 probably less than 100 people worldwide) that <g> IS A MUCH BETTER INDICATOR OF THE EXTENT TO WHICH A TREATMENT IS EFFECTIVE THAN IS EITHER GAIN OR POSTTEST, for example, if the treatment yields <g> > 0.3 for a mechanics course, where <g> is calculated from pre/post testing with the "Force Concept Inventory" (Hestenes et al. 1992) then the course could be considered as in the "interactive-engagement zone" (Hake 1998a).
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
V. IS PRE/POST TESTING BECOMING MORE POPULAR IN DISCIPLINES OTHER THAN PHYSICS?
The answer is "YES." The relatively early pre/post test work of:
(a) Halloun & Hestenes (1985a,b) in developing the "Mechanics Diagnostic" test (precursor to the much-used "Force Concept Inventory" (FCI) [Hestenes et al. (1992)]);
(b) Hake (1987) in physics;
(b) Paden & Moyer (1969) in economics;
(c) Sundberg & Moncada (1994) in biology;
(d) Milford (1996) in chemistry; and
(e) Zeilik et al. (1997) in astronomy.
has been followed by efforts to use valid and consistently reliable such as the Force Concept Inventory (FCI) [Hestenes et al. (1992) in a pre/post mode so as to formatively measure the need for, and the results of, reform methods in:
(1) physics [for reviews see Hake (2002a,b)];
(2) biology [e.g., Lawson (2001), Anderson et al. (2002), Klymkowsky et al. ( 2003), Roy (2001, 2003), Sundberg (2002), and Wood (2003);
(3) chemistry [e.g., Bowen & Bunce (1997), ASU (2003), Gonzalez et al. (2003)];
(4) engineering [e.g., Evans et al. (2003)]; and
(5) computer science [e.g., Almstrum (2003)].
NOTE THAT MATHEMATICS IS ABSENT FROM THE ABOVE LISTING. . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
