For those who are interested: Teachers College Record (http://www.tcrecord.org), Volume 1 for 2005 discusses the use of qualitative, and other non-RCT approaches to assessment.
Annette Quoting Richard Hake <[EMAIL PROTECTED]>: > If you object to cross-posting as a way to tunnel through inter- and > intra-disciplinary barriers, please hit "delete" now. And if you > respond to this long (18 kB) post, please don't hit the reply button > unless you prune the original message normally contained in your > reply down to a few lines, otherwise you may inflict this entire post > yet again on suffering list subscribers. > > Before proceeding to the main substance, I should like to relay an > important comment from EdStat's Dale Berger (2005): > > "Those interested in the 'Gold Standard' issue may wish to read or > see the 'debate' between [psychologist] Mark Lipsey and [philosopher] > Michael Scriven last summer. It is available in text or streaming > video through the link <http://www.cgu.edu/pages/2668.asp>." > > In his AERA-C post of 18 Apr 2005 with the above title, Dennis > Roberts (2005) wrote [bracketed by lines "RRRRRR. . . ."]: > > RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR > At 08:54 AM 4/18/2005 [Jason Osborne (2005)] wrote: > > "I have been amazed that this conversation has been going on as long > as it has . . . and the lack of fiery rebuttal bothers me as well. > While I agree that RCTs are not ALWAYS appropriate, as Richard. . . > .[Hake (2005a)]. . . suggests, they are and should be the gold > standard in terms of creating STRONG inference." > > ON the other side of the coin ... it also amazes me when people can't > see that while education is not perfect by any stretch of the > imagination ...PRODUCTS from this system in large part are highly > intelligent ... invent things ... make scientific discoveries ... > solve medical problems ... lead governments ... fly 767s ... orbit > the planet ... build the Golden Gate Bridge ... and just a zillion > other accomplishments THAT for the most part ... started back in > their school days. > > School, w/o RCT, does make a huge difference in peoples' lives > ...While it might be nice when the opportunity arises to do more RCT > in school settings ... it's not the gold standard. Look at the > accomplishments of students as they become adults ... THAT's the > proof. > RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR > > Great minds run in the same direction. Dennis's point that, judging > from its products, the old-time education was not so bad, was also > made by POD's Mike Theall (2005a) who wrote [bracketed by lines > "TTTTTT. . ."; my insertion of a,b,c] : > > TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT > a. For the majority, education does a pretty good job. . . . > > b. . . . while all the critics scream for "reform" in education, we > don't hear much screaming about reforming anything else. Our business > & political critics need to do a lot of housecleaning before casting > any stones. Law, medicine, and other professions aren't perfect > either and rather than accepting any responsibility for their > mistakes, they simply point to education as a handy scapegoat. > Frankly, I'm tired of the hypocrisy. . . . . > > c. . . . We all know and admire scores of teachers who are dedicated, > thoughtful, capable, and hard-working. Let's remember these people > in our statements and let's fight for the recognition and credit they > and education deserve.. . . . > TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT > > My response [Hake (2005b)] was (with apologies to those who have seen > this previously): > > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH > I enthusiastically agree with Mike Theall's points "b" and "c," and > with Mike Chejlava's (2005) comment regarding the reality of C.P. > Snow's "culture gap," so apparent in the posts of Theall (2005), > Chejlava (2005), and the POD discussion list generally. > > However, I vehemently disagree with Mike Theall's point "a," that > > "FOR THE MAJORITY, EDUCATION DOES A PRETTY GOOD JOB." > > In Hake (2000) I listed a few examples of the science illiteracy of > the general population (and even the elite graduates of Harvard and > MIT) for whom education has NOT done a "pretty good job" (see that > article for the references): > > EVIDENCE FOR SCIENTIFIC ILLITERACY (a few of many examples) > > A. Science and Engineering Indicators (NSF, 1998), Chap. 7, "Science and > Technology"; on the web at > <http://www.nsf.gov/sbe/srs/seind98/start.htm>: "..... it appears > that only 11 percent of Americans can define the term 'molecule.' . . > . A large proportion of the population knows that a molecule is a > small piece of matter, but is unable to relate it to an atom or a > cell, which are also small pieces of matter. And, despite substantial > media attention to deep space probes and pictures from the Hubble > Space Telescope, only 48 percent of Americans know that the earth > goes around the sun once each year .. Only 27 percent of Americans > understand the nature of scientific inquiry well enough to be able to > make informed judgments about the scientific basis of results > reported in the media. Public understanding of the nature of > scientific inquiry was measured through questions about the meaning > of scientific study and the reasons for the use of control groups in > experiments." > > B. Eugenie C. Scott, "Not (Just) in Kansas Anymore," Science 228, > 813-815 (2000); on the web at > <http://www.sciencemag.org/cgi/content/full/288/5467/813>: "In August > of 1999, after months of wrangling, the Kansas State Board of > Education passed its state science education standards. Against the > recommendations of a committee of 27 scientists and teachers, the > board voted to strip from the standards all mention of the Big Bang, > the age of the Earth, and any reference to organisms having descended > from the same ancestors: in other words evolutionary astronomy, > geology, and biology. Teachers were informed that evolution would not > be included in the state high-school assessment exams, greatly > decreasing the likelihood that the subject would be taught . . . As > the media probed for more stories, the National Center for Science > Education (where I work. . . <http://www.natcenscied.org/> . . .) > informed the sometimes incredulous press that, yes, indeed, > antievolutionism is a widespread problem in American kindergarten > through high school or 'K-12' education." > > C. Jerome Epstein, "Cognitive Development in an Integrated > Mathematics and Science Program," J. of College Science Teaching, > 12/97 & 1/98, pp. 194-201: > "While it is now well known that large numbers of students arrive at > college with large educational and cognitive deficits, many faculty > and administrative colleagues are not aware that many students lost > all sense of meaning or understanding in elementary school. . . .In > large numbers our students. . .[at Bloomfield College (NJ) and Lehman > (CUNY)] . . . cannot order a set of fractions and decimals and cannot > place them on a number line. Many do not comprehend division by a > fraction and have no concrete comprehension of the process of > division itself. Reading rulers where there are other than 10 > subdivisions, basic operational meaning of area and volume, are > pervasive difficulties. Most cannot deal with proportional reasoning > nor any sort of problem that has to be translated from English. Our > diagnostic test, which has been given now at more than a dozen > institutions shows that there are such students everywhere. . . > .(even Wellesley! - see J. Epstein, "What is the Real Level of Our > Students," 1999, unpublished). > > D. Woodie Flowers "Why change, Been doin' it this way for 4000 > years!" ASME Mechanical Engineering Education Conference: Drivers and > Strategies of Major Program Change, Fort Lauderdale, Florida, March > 26-29, 2000; on the web as PowerPoint plus video at > <http://hitchcock.dlt.asu.edu/media2/cresmet/flowers/>. (Download the > free RealPlayer.): > > Slide 42, 43: BBC videotape Simple Minds showing MIT graduates having > trouble getting a flashlight bulb to light, given one bulb, one > battery, and one piece of wire. This is the MIT counterpart of > Harvard's "A Private Universe," a videotape showing Harvard > graduating seniors confidently explaining that the seasons are caused > by yearly variation in the distance of the Earth from the Sun! And if > such occurs at MIT and Harvard, how about Podunk State? > For "A Private Universe" consult > <http://www.learner.org/resources/series28.html>. For an equivalent > of the Simple Minds videotape go to Minds of Our Own at > <http://www.learner.org/resources/series26.html>. The latter gives a > description of three Minds of Our Own videotapes available, including > the MIT footage: "Why is it that students can graduate from MIT and > Harvard, yet not know how to solve a simple third-grade problem in > science: lighting a light bulb with a battery and wire? Through the > example of an experienced teacher, the program takes a hard look at > why teaching fails, even when he uses all of the traditional tricks > of the trade. The program shows how new research, used by teachers > committed to finding solutions to problems, is reshaping what goes on > in the nation's schools." > > But why should one be concerned about the science illiteracy of the > general population? Because life-threatening science-related societal > problems [e.g., terrorism; overpopulation (doubles about every 35 > years); threat of weapons of mass destruction; human welfare > (starvation, homelessness, unemployment, > drugs, epidemics, AIDS, etc.); pollution of air, water, land, food; > global warming; ozone depletion; deforestation; loss of biodiversity; > etc., etc., etc.] cannot be resolved when a scientifically illiterate > society elects scientifically illiterate leaders. > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH > > Richard Hake, Emeritus Professor of Physics, Indiana University > 24245 Hatteras Street, Woodland Hills, CA 91367 > <[EMAIL PROTECTED]> > <http://www.physics.indiana.edu/~hake> > <http://www.physics.indiana.edu/~sdi> > > "The global population is precariously large, and will become much > more so before peaking some time after 2050. Humanity overall is > improving per capita > production, health, and longevity. But it is doing so by eating up > the planet's capital, including natural resources and biological > diversity millions of years old. Homo sapiens is approaching the > limit of its food and water supply. Unlike any species before, it is > also changing the world's atmosphere and climate, lowering and > polluting water tables, shrinking forests, and spreading deserts. > Most of the stress originates directly or indirectly from a handful > of industrialized countries. Their proven formulas for prosperity are > being eagerly adopted by the rest of the world. The emulation cannot > be sustained, not with the same levels of consumption and waste. Even > if the industrialization of the developing countries is only > partially successful, the environmental aftershock will dwarf the > population explosion that preceded it." > E.O. Wilson in "Consilience: The Unity of Knowledge" (Knopf, 1998) > > > REFERENCES > Berger, D. 2005. "Re: Should Randomized Control Trials Be the Gold > Standard of Educational Research? online at > <http://lists.psu.edu/cgi-bin/wa?A2=ind0504&L=edstat- l&T=0&O=D&X=5892603D63E947357B&Y=rrhake%40earthlink%2Enet&P=6741>. > > Post of 18 Apr 2005 21:52:50-0700 to EdStat. The encyclopedic URL > indicates that EdStat is one of the few discussion lists whose > archives are closed to non subscribers :-( - WHY ??. However, it > takes only a few minutes to subscribe by following the simple > directions at > <http://lists.psu.edu/archives/edstat-l.html> / "Join or leave the > list (or change settings)" where "/" means "click on." If you're > busy, then subscribe using the "NOMAIL" option under "Miscellaneous." > Then, as a subscriber, you may access the archives and/or post > messages at any time, while receiving NO MAIL from the list! > > Chejlava, M. 2005. "Re: Random Thought: Leave Them Asking," POD post > of 21 Feb 2005 11:07:14-0500; online at > <http://listserv.nd.edu/cgi-bin/wa?A2=ind0502&L=pod&O=D&P=25059>. > > Hake, R.R. 2000. "The General Population's Ignorance of Science > Related Societal Issues: A Challenge for the University," AAPT > Announcer 30(2): 105; online as ref. 11 at > <http://www.physics.indiana.edu/~hake> or download directly by > clicking on <http://www.physics.indiana.edu/~hake/GuelphSocietyG.pdf> > (2.1MB). > Based on an earlier libretto with the leitmotiv: "The road to U.S. > science literacy begins with effective university science courses for > pre-college teachers." The opera dramatizes the fact that the failure > of universities THROUGHOUT THE UNIVERSE to properly educate > pre-college teachers is responsible for our failure to observe any > signs of extraterrestrial intelligence. > > Hake, R.R. 2005a. Re: Should Randomized Control Trials Be the Gold > Standard of Educational Research? online at > <http://lists.asu.edu/cgi-bin/wa?A2=ind0504&L=aera-l&T=0&O=D&P=2100>. > Post of 17/18 Apr 2005 to AERA-C, AERA-D, AERA-G, AERA-H, AERA-J, > AERA-K, AERA-L, > AP-Physics, ASSESS, Biopi-L, Chemed-L, EvalTalk, Math-Learn, Phys-L, > PhysLrnR, Physhare, POD, STLHE-L, & TIPS. > > Hake, R.R. 2005b. "Attacks on Education (was Random Thought: Leave > Them Asking)" online at > <http://listserv.nd.edu/cgi-bin/wa?A2=ind0502&L=pod&P=R20579&I=-3>. > Post of 21 Feb 2005 10:33:05-0800 to AERA-J, AERA-L, AP-Physics, > Biopi-L, Chemed-L, PhysLrnR, Phys-L, Physhare, Physoc, & POD. > > Osborne, J. 2005. "should randomized control trials be the gold > standard?" online at > <http://lists.asu.edu/cgi-bin/wa?A2=ind0504c&L=aera-c&D=0&T=0&P=467&F=P>. > Post of 18 Apr 2005 08:54:11-0400 to AERA-C. > > Roberts, D. 2005. Re: should randomized control trials be the gold > standard? online at > <http://lists.asu.edu/cgi-bin/wa?A2=ind0504c&L=aera-c&T=0&O=D&P=598>. > Post of 18 Apr 2005 10:47:40-0400 to AERA-C. > > Snow, C.P. 1959. "The two cultures and the scientific revolution." > Available in a 1993 "Canto" edition tiled "The Two Cultures," > illustrated by Stefan Collini and published by Cambridge University > Press. The publisher states "This reissue of Snow's controversial > Rede lecture of 1959 and its successor piece 'A Second Look' has a > new introduction that charts the history and context of the famous > debate on the cultural split between the humanities and the sciences." > > Theall, M. 2005a. "Re: Random Thought: Leave Them Asking," POD post of > 21 Feb 2005 08:30:48-0500, online at > <http://listserv.nd.edu/cgi-bin/wa?A2=ind0502&L=pod&O=D&P=24726>. For > Theall's response to Hake (2005b) see Theall (2005b). > > Theall, M. 2005b. "Re: Attacks on Education (was Random Thought: > Leave Them Asking)," POD post of 21 Feb 2005 15:27:38-0500, online at > <http://listserv.nd.edu/cgi-bin/wa?A2=ind0502&L=pod&P=R21044&I=-3>. > > --- > You are currently subscribed to tips as: [EMAIL PROTECTED] > To unsubscribe send a blank email to [EMAIL PROTECTED] > Annette Kujawski Taylor, Ph. D. Department of Psychology University of San Diego 5998 Alcala Park San Diego, CA 92110 [EMAIL PROTECTED] --- You are currently subscribed to tips as: [email protected] To unsubscribe send a blank email to [EMAIL PROTECTED]
