In view of recent discussion regarding IQ tests the following may be of interest to some FWers. I don't know where and when it was published but I found it interesting because although I was aware of the SAT test in the US (much superior to our A level system, in my view) I didn't know a great deal about it. (I presume this article was published fairly recently.)
Keith Hudson <<<<<<<<<< The SAT Revolution by Julian E. Barnes The new test spells the end of IQ—and big changes for American education In a cramped basement classroom in Berkeley, Calif., Kim Nunlist is unraveling the mysteries of the SAT. Nunlist, a 22-year-old undergrad with a serious tattoo and a score of 1470, waves the College Board's book of tips and practice tests in front of her class. Rip out the first 275 pages that explain how to answer SAT questions, she commands. "This is by the people who own the test; they do not want you to score well," she tells the class. "Do not pay attention to what they say. Pay attention to what I say." Near the front of the room, Eryn Leavens stares intensely at a Princeton Review study guide. A high school senior, Leavens is determined to succeed on the standardized tests she despises. Nunlist asks her the answer to a sentence-completion item, in which she was asked to choose words to fill the blanks in a sentence. "Although on the surface the final draft appeared to . . . . . the first draft, upon close inspection it was . . . . . that major changes had been made." Leavens hesitates, then answers. "I picked C, but it didn't sound right." Nunlist agrees that the answer—"reproduce" and "apparent"—sounds weird, but it is correct. "They give you bad words for that reason," she says. "You have to pick the best answer. The best answer often stinks." Nunlist has no illusions that what takes place in her classes can be called education. "It is bunk," she says. "I teach how to take a multiple-choice test." And what she teaches, she adds, is "completely inapplicable to the rest of life." This is just what Carl Brigham feared. When Brigham, a psychologist, created the Scholastic Aptitude Test in 1926, he was a firm believer in intelligence testing. With IQ tests, he thought, he would be able to predict applicants' college grades, thereby selecting the students who would benefit most from higher education. But as his thinking evolved, Brigham began to worry that his test would lead teachers to focus on "linguistic skills" rather than literature and "disintegrated bits" of computation instead of mathematical concepts. His concerns went unheeded, and in the decades that followed the SAT only grew in power, becoming the pre-eminent gatekeeper for American higher education—as well as the stuff of sleepless nights for many a high schooler. Now the era of intelligence testing is about to end. Thanks to an unprecedented assault from the head of the University of California system, the College Board (the nonprofit organization that owns the SAT) has begun its biggest overhaul ever of the test. The 340,000 students who took the SAT last weekend saw the same old kinds of questions. But by 2005 the board plans to strip out the analogies section, ask questions based on more-advanced math, and add a grammar and essay-writing test. "In many respects," says Ida Lawrence, the SAT program director, "it is a revolution." Although the College Board's announcement in July was front-page news, the significance of the changes has remained largely unexamined. The inside story of the battle that began in California reveals just how great a philosophical shift the College Board has embraced. Rather than assess raw intelligence, the new SAT is intended to measure academic preparedness. "In its original form it was an IQ test," says Gaston Caperton, the College Board president. "What we have done is take the SAT and make it into something that tests reasoning and developed skill." What they have done is taken hold of the diseased American education system at its root. For with these new changes, the SAT will effectively set education standards for the nation's high schools. But can the College Board do what a nation of education reformers couldn't? America is put to the test In 1901, 978 young men and women applying to Columbia, Barnard, and New York University were the first to take"College Boards." In those early days, students sat for a series of grueling essay tests composed by college and high school teachers, in subjects including chemistry, Latin, history, mathematics, and physics. To complete the English exam, for instance, students had to write intelligently about books chosen from an assigned list of classics including Macbeth, The Last of the Mohicans, and Silas Marner. This was still the dawn of American public education. Only about 6 percent of 17-year-olds in 1900 graduated from high school. Only 2 percent would go on to graduate from college; few of the rest of the nation's teenagers even aspired to do so. Most of those who went were white, wealthy, Protestant, and male, and the early entrance tests bespoke this world of privilege. Some colleges had specific reading lists, some had unique exams, and some admitted students on recommendations by private-school headmasters alone. To get into Yale, students had to take oral exams to test their mental "power." The College Board was founded not just to standardize widely varying entrance requirements but also to shape what was taught in high school. "The College Board was the standard-setter for American education," says educational historian Diane Ravitch. "[It] said 'these are the works we want students to study.' " The SAT promised to be just the opposite of those old College Boards. It was designed to measure innate intelligence, not what students had learned in school. The first champion of the SAT, Harvard University President James Bryant Conant, started a scholarship program in 1934 to bring bright men from humble Midwestern backgrounds to his college. Conant, according to Nicholas Lemann's 1999 book The Big Test, believed achievement tests like the College Boards favored wealthy students from fancy private academies. So to select his new scholars, Conant decided to use the test that Brigham had designed a few years earlier. Princeton and Yale joined in with similar scholarships that used the SAT. But it wasn't until World War II that the College Board was forced to drop its essay tests, which took too much time and too many graders, and adopt the SAT for all students. That was fine with Conant, not because he wanted to expand access to higher education; he believed that too many people already went to college. What he wanted to change was who got to go. He wanted to create a system where the country was led not by the children of privilege but by those with the greatest natural intelligence. The shift was not without consequences, says Ravitch: When the College Board gave up the essay tests for the SAT, she says, it also abandoned its oversight of what high schools taught. The next few decades were the heyday of intelligence testing. Educators believed the tests would improve teaching by efficiently sorting students into different ability groups. By the 1960s, almost all public-school children were given a group-administered IQ test. But the tests always had critics who were skeptical that intelligence was fixed or even measurable. Because blacks, as a group, scored lower than whites on the tests, some opponents argued that the assessments really reflected economic disadvantage, not intellectual difference. Court decisions in the 1970s reinforced the criticism, restricting the use of IQ and tarnishing the image of intelligence testing. "There is some deep sense where IQ seems un-American," says Christopher Jencks, a Harvard professor and coeditor of The Black-White Test Score Gap. "It may be real, but it is not something we want to make a big fuss about." Today, individual IQ tests are still used to assess learning disabilities or measure mental retardation. But few public schools give group-IQ tests anymore, and educators are dumping the last of them to make way for new federally mandated achievement tests. The Educational Testing Service, which was created to write and administer the SAT for the College Board, insists that its exam ceased being an IQ test as far back as 1946. That was the year many logic items were replaced with questions based on reading passages. Despite those changes, question styles that are staples of IQ tests, like analogies, remained on the SAT, and many students, parents, and teachers still think of the SAT as the last mass-administered intelligence exam. "I think the SAT tests cleverness," says Danny Jaye, the head of the mathematics department at New York City's prestigious Stuyvesant High School. "It is not a content test as much as it is an IQ test." Measuring up, measuring down In fact, there is no consensus on what the SAT does measure. Though it was supposed to democratize American universities by sidestepping the hereditary aristocracy of wealth and replacing it with a fluid meritocracy of talent, that's not how things worked out, according to critics like Harvard University testing expert Howard Gardner. His recent experience using coaching guides to help his 17-year-old son, Benjamin, study for the SAT has underscored his view that test prep corrupts what the exam is supposed to gauge. Gardner says that when he took the SAT in the1960s it was "an intelligence test skewed to measure the ability to get into college." "Now," he says, "it measures how good your tutor is." The College Board says the SAT is a measure of students' ability to reason. "We want people to think out of the box," says Wayne Camara, a College Board vice president. By assessing students' ability to solve problems quickly, the board argues, the SAT helps predict freshman grades, thereby giving admissions officers a valuable tool . "If you are trying to forecast what people will do in college," says Lawrence, the SAT's program director, "you need a test of reasoning." But critics say that achievement tests like the SAT II writing exam are just as good or better predictors of college grades. "Reasoning," in their view, is just a euphemism for IQ. John Katzman, the founder of the Princeton Review test-prep company, contends that the SAT measures a very narrow kind of reasoning, the kind of mental quickness that is useful for solving crossword puzzles. The Princeton Review says that because the SAT does not measure the kind of problem solving that schools teach, the best way to prepare for the test is to memorize vocabulary and learn perfect exam-taking strategies. The company, which last year took in $69 million in revenue while making that argument, acknowledges that its lessons are educationally vacant. But that is not the company's fault, says Jeff Rubenstein, a Princeton Review assistant vice president. "The SAT inspires dysfunctional educational behavior," Rubenstein says. "Students shouldn't be memorizing words like lummox or malinger. They should be reading Dickens or Shakespeare." In her Princeton Review class, Nunlist builds up her students' confidence by tearing down the College Board and ETS. On her whiteboard, she writes: "If 4x + 6 = 30, then 2x equals . . . ?" She turns to the class and asks, "Lauren, what did you get?" Lauren Steinberg looks at her workbook. "I got C," she says. C is 6. Nunlist cracks a half smile. "That is a prime way ETS will mess you up," she responds. "You are trained to solve for x so they ask you for 2x." The answer is D: 12. Steinberg, a 17-year-old senior at Albany High in Kensington, Calif., has a 1240 on the SAT. She wants to go to Berkeley, but her father worries her score is not high enough. So he enrolled her in Nunlist's $1,000 class, where Steinberg learns to master test tricks and hate the SAT. "It doesn't tell what you know," Steinberg says after her second week in class. "They are trying to trick you." If she's right, the trick's on ETS, because all this test strategizing may be making the SAT less good at predicting how students will do in college. University of California data show that achievement test scores are slightly better than SAT scores at predicting grades for the first year of college. Some researchers suspect the culprit is test prep. Wealthier students often get lots of coaching on the SAT (but not on the achievement tests, which typically cover the material students have learned in, say, freshman biology or junior-year U.S. history). "You can teach students how to raise their score on the SAT, but that is not improving their abilities or increasing their knowledge," says Michael Brown, a member of the University of California faculty committee that examined the SAT. "But to do better on an achievement test, you have to know the material, and preparation will entail increasing students' knowledge of the subject matter." A glimpse of an alternate test-prep world could be seen in a course given by Princeton Review competitor Kaplan on a recent evening in midtown Manhattan. For most of the class, Meredith Moore, the teacher, reviewed the usual tricks. But when she turned to the critical-reading passages—which will grow more numerous on the new SAT—the tone changed. As the class read the passage about the discovery of penicillin together, Moore quizzed the students on why the author included a particular detail. They couldn't simply recite a time-honed test trick or memorized vocabulary word; they had to really understand what they had read. Viktor Hristov, a 17-year-old Bulgarian immigrant who lives in Astoria, Queens, signed up for the $900 course, hoping to raise his 1050 on the SAT by 300 points. But, he says, the classes are also helping him build longer-lasting skills. "I am learning to read faster and understand it better," he says. In other words, test-taking is inspiring him to learn something that is actually worth learning in the first place. The revolution from without One night in the fall of 1999, Richard Atkinson sat down to take the SAT. As president of the 174,000-student University of California system, Atkinson wanted to learn more about the test that helped determine who would study at his school. One of the nation's most respected cognitive psychologists, he had a long interest and expertisein educational testing. In the 1960s, in fact, Atkinson had helped found a company that produced computer-based tests for elementary-school children. The next morning, Atkinson called his new deputy, Patrick Hayashi, into his office. "What the hell are these analogies?" Atkinson demanded. Thwack. He slammed his hand on the table: "What theory of cognitive development justifies this?" Hayashi, who had overseen admissions at Berkeley for 12 years, knew the SAT well. But he was reluctant to debate Atkinson or interrupt his rant. Hayashi held his tongue, and Atkinson pressed forward. "The SAT is based on concepts of intelligence no one holds anymore," he said. "This is a test based on the idea intelligence is immutable." Atkinson did not like what the SAT seemed to measure, and he hated the behavior it inspired. He and Hayashi, who would soon become a College Board trustee, hit on a revolutionary idea. They knew that using an intelligence test led to drills in test tricks. So it followed that using a test that measured students' knowledge of a rigorous high schoolcurriculum would force schools to make courses more challenging. They believed that by changing their university's admissions test they could change the behavior of both high school students and teachers. And they believed that dumping the SAT in favor of achievement tests would improve teaching at all California high schools. Atkinson's skepticism about the SAT solidified when he learned that his 12-year-old granddaughter was being drilled in SAT analogies at school. A few months later, in February 2001, he gave a speech calling on the University of California to drop the SAT in favor of achievement tests. At first, the College Board was skeptical about Atkinson's motives. Because Latino students tended to do well on the Spanish language exam, their achievement scores were better than their SAT marks. Some SAT defenders thought Atkinson was pushing for the achievement tests just to increase the numbers of Hispanic students at Berkeley and UCLA—an end run around the University of California regents' 1995 banon affirmative action. The College Board remained suspicious of Atkinson's motives until the two sides met at a November 2001 conference in Santa Barbara. There, Saul Geiser, Atkinson's research director, used university data to demonstrate that switching to achievement tests would not substantially change the racial makeup of UC's student body. ETS, College Board, and California researchers agreed that when used with high school grades, the SAT and SAT II achievement tests had a nearly equal ability to help predict college grades. But Geiser went further, saying that the reason to use achievement tests went beyond predictive power. "They are the best chance we have of developing a rigorous high school curriculum where it does not exist," he said at the conference. After the meeting, the College Board came to see Atkinson's proposal as education reform, not racial gerrymandering. "That established a common cause," says James Montoya, a College Board vice president. "We all wanted to connect the SAT to what students are supposed to learn in school." There was more to the College Board's change of heart than just a meeting of the minds. After Atkinson recommended dropping the SAT, a University of California faculty committee began talking with College Board officials about creating a new achievement test just to serve their state. As talks progressed, the committee became concerned that Stanford, Harvard, Yale, or other elite private schools might not accept a test unique to California. So they began to look closely at the ACT, a competitor to the SAT that was accepted by the University of California but little used by the state's students. "We were very interested in the ACT," says Dorothy Perry, the former committee chair. "They were very close to what we felt a test should be." The ACT, founded in 1959 as the American College Testing Program, is the College Board's great rival and ideological opposite. The SAT measures students' ability to learn. The ACT measures what students have already learned. Though most colleges accept both, the SAT dominates the coasts and the ACT reigns in the heartland. Today, the ACT's developers regularly alter their test to match what schools teach. And some states, like Illinois, have begun to use the ACT to judge whether students have mastered state learning standards. That has led schools in Vienna, Elgin, and other Illinois towns and cities to take steps including beefing up their curriculum, adding extra reading-skills classes, and pushing students to take more-difficult classes earlier. That is precisely the kind of educational behavior Atkinson believes admissions tests should inspire. The College Board had no intention of letting California become an ACT state. Californians make up 12.6 percent of the 2.2 million annual SAT test takers—a sizable chunk of the $141 million the College Board takes in from its college admissions tests each year. Faced with the possibility that its most important customer would leave for the competition, the College Board caved and began to talk about changing the entire SAT. The group also came to see reform as a way to do more than just keep California, says Hayashi, the Atkinson deputy and College Board trustee. For a decade, the board had tried to change how people thought of the test by fiddling with the name. First it briefly changed the "A" in SAT from "aptitude" to "assessment." Then it declared that the initials stood for nothing at all. "No matter what they did," Hayashi says, "thelegacy of the SAT as an aptitude test was a millstone that hung round their neck. This was a chance to get rid of it." In June, the College Board trustees voted to drop analogies and quantitative comparisons, a question style that makes students look at two columns of numerical information and decide which is larger. The board said it would add more long passages to the verbal section and rename it "critical reading." A revamped math section would test more advanced coursework. Finally, the College Board decided to add a version of the writing achievement test, which includes an essay and multiple-choice grammar questions. A perfect score would rise from 1600 to 2400. Atkinson declared victory. Diamonds in the rough At Fremont High School in Oakland, Calif., not even teachers show up regularly for class. Most instruction consists of giving students worksheets and telling them to keep quiet. Only a third of entering freshmen graduate from Fremont in four years, and few of those go on to four-year colleges. And yet there are Advanced Placement classes here, where kids learn organic chemistry and read Richard Wright's Native Son. And these kids want to go to college. Like the four students who are sitting in a classroom trailer anchored in the Fremont schoolyard one day, talking about the SAT. No different from students around the country, these four worry that college admissions officers will place too much emphasis on the SAT. "When I took it as a freshman, the analogies confused me," says Marlene Labastida, a senior. For these students, AP classes are like a life raft in an ocean of failure. Though critics have long complained that a reliance on the SAT keeps out good students from bad schools, the tests' supporters say this is where the test is most needed. Looking at kids from schools like Fremont, college admissions officers often have a hard time judging whether an A average means the student is prepared for college-level work. That is where the SAT comes in. Labastida has a 1260, putting her at the 86th percentile nationwide and substantially ahead of her best classmates. Labastida's English teacher, Daniel Hurst, says her testscore reflects real intellectual ability. "Marlene has a chance in life," he says. "There are always one or two absolute gems, like Marlene, who come in having read their whole lives, and they have a wonderful facility for language." Despite California officials' certainty about the superiority of curriculum-based tests, some skeptics believe any alteration in the SAT could end up hurting applicants like Labastida. After the changes, the test may no longer spotlight "the diamond in the rough," says Rebecca Zwick, a former ETS researcher and author of Fair Game?, a new book about standardized testing. "There is a risk that by incorporating more advanced math, for example, it will make the test more sensitive to differences in schooling." Too often, she adds, "kids who go to school in poor areas do not have access to competent instruction." California officials maintain that the idea that the SAT has a unique ability to find a diamond in the rough is a myth. For one thing, about 71 percent of students earn similar marks on the SAT I and on the writing and math achievement tests, according to the College Board. Labastida, for example, scored a 650 out of 800 on her writing achievement test and a 4 out of 5 on her AP History test. And according to California research, the SAT II achievement tests are actually better at finding diamonds in the rough than the SAT. Low-income students, overall, do better on the writing and math achievement tests than on the SAT. Geiser, the California researcher, compared students with high SAT scores and low writing and math achievement marks with those who scored high on achievements but poorly on the SAT. He found that the students in the second group had better high school grades, better college grades, and generally came from less wealthy backgrounds. In addition, those students who were admitted to UC with high SATs but low high school grades ended up earning generally poor marks in college. "Lazy underachievers in high school are lazy underachievers in college," Geiser says. Even if the critics are right, Atkinson argues, designing a test just to find undiscovered "diamonds" is too narrow a goal. Instead, he says, the purpose of admissions tests should be to deliver the greatest high school educational benefit to the majority of American kids. "I want schools to improve," Atkinson says. "There are core ideas I want everyone to have mastered. . . . We need to get schools to teach the right curriculum." The future of testing But even if the College Board succeeds in getting American schools to change what they teach, is that the best method of reform? Using tests to shape curriculum inevitably narrows what is taught in school, says Christina Perez of the anti-testing group FairTest. "I am much more comfortable with universities and high schools deciding what students should learn rather than an external organization like the College Board," she says. "If schools focus on the material in the SAT, then they are missing out on a curriculum tied to students' lives and reflective of our society." But like the College Board of a century ago, the leaders of today's organization once again believe they can use their admissions test to improve schools. "We have an uneven education system in this country, and we have to focus on making that better," Caperton says. When the latest national SAT averages were released in late August, he took pains to underline the connection between what high schools teach and what the SAT easures. Although math marks rose, verbal scores were stagnant. If performance is to improve, Caperton said, schools need to start teaching more writing and more grammar. It was a newly aggressive position for the College Board, one it would not have takenwithout Atkinson's push. In September, a little more than a year and a half after Atkinson made his speech attacking the SAT, the University of California held a conference at Berkeley to tell guidance counselors about the changes to the test and college admissions. Wearing a blue blazer and turquoise-clasped bolo tie, Montoya, the College Board vice president, told the counselors they should take a new view of the test. "What we want," he said, "is that at the end of this all, when someone thinks of their SAT score, they do not relate it to being smart or not smart, but . . . to being prepared or unprepared." Beth Pascal, a counselor from Burlingame High School near San Francisco, sat near the front of the lecture hall looking skeptical. Despite the new rhetoric surrounding the overhauled exam, she says, "a lot of kids feel it is a measure of their intelligence." It will be tough to convince students that the new SAT is no longer a gauge of IQ. Reformers hope students see that the changes mean they can boost their scores—and reduce their testing anxiety—by taking rigorous classes, honing important life skills, and demanding more from their schools. Now, says William Fitzsimmons, a College Board trustee and dean of admissions at Harvard, "the pressure will be on schools."And if schools respond, the much-feared SAT will have done somethingto improve education for everyone. >>>>>>>>>> ---------------------------------------------------------------------------- ------------ Keith Hudson, General Editor, Handlo Music, http://www.handlo.com 6 Upper Camden Place, Bath BA1 5HX, England Tel: +44 1225 312622; Fax: +44 1225 447727; mailto:khudson@;handlo.com ________________________________________________________________________
