May 24, 2000
MANAGEMENT
Makes Sense to Test for Common Sense. Yes? No?
By DAVID LEONHARDT
he story of the uninspired student
who goes on to make a name for himself is practically a cliché. Winston
Churchill was a notoriously poor test-taker. Martin Luther King Jr. scored
below average on every section of the
graduate school boards, including verbal aptitude. Like many other corporate leaders, Richard Branson of the
Virgin Group had trouble with standardized admission tests.
It is a paradox that has long frustrated educators. They know that test
scores often predict applicants' academic performance on campus but are
unreliable guides to career success. As
a result, psychologists and corporate
recruiters alike have longed for a systematic way to identify the campus
goof-off who is destined to become a
corporate chief executive or the class
brain who is doomed to languish in middle management.
Now, for what may be the first time,
a major university is using its own students and applicants in a concerted effort to unlock the mystery. In a challenge to the primacy of the standardized tests that millions of Americans
have sweated over, the University of
Michigan Business School is developing
a test of practical intelligence, or common sense, that it hopes will do a better
job of identifying future leaders than
the Graduate Management Admission
Test, or G.M.A.T., does.
Michigan's new test leads students
through an elaborate series of business
scenarios that present hypothetical financial statements, press releases,
news articles and other information.
It then poses a central problem and
asks students both open-ended and
multiple-choice questions: What do
you see as the main problem in this
situation? What information did you
focus on? What obstacles, if any, do
you anticipate?
The test aims to gauge who is able
to learn from mistakes, handle
changing situations and cope with
less-than-perfect information -- the
same challenges, its designers say,
that working people face every day.
Michigan officials gave the test to
M.B.A. students entering last fall,
and they plan to do so again this
year. The school will then compare
the students' performance on the test
to their track records at the school,
from grades to extracurricular leadership positions to job offers, to see
whether it is a valid measure of their
ability to function in the workplace.
Once the correlations are more
clear, Michigan intends to require
applicants to take the test, possibly
as soon as the fall of 2001, in addition
to the G.M.A.T.
"We need a better, fairer, broader
way of measuring people," said B.
Joseph White, the dean for the last
decade at Michigan, which is one of
the country's most highly ranked
business schools. "We want to create
an entirely new method of assessing
incoming students for our M.B.A.
program in order to spot people who
are going to have the highest probability of being successful."
Mr. White says his ultimate goal is
to create a yardstick that will become a standard tool for admissions
officers at business schools around
the country. "There's some X factor
that is critical to people's success
that I don't think we or any top business school is measuring," he said.
Outsiders say Michigan is fighting
a difficult battle, even though some
of its largest corporate recruiters
say they think the test could be useful to them. Michigan could scare
away some applicants by requiring
them to take a second standardized
test. And some people are skeptical
that any form of test -- as opposed to
an interview, for example -- can
measure one's ability to interact
with others, let alone the many variables that contribute to professional
success. Determining the validity of
such a test could take years and
could be costly.
"I certainly think it's a good endeavor, but it's very tough" to develop a new test, said Paul Danos,
dean of the Tuck School of Business
at Dartmouth.
Still, the effort is a boon to the
group of educational psychologists
who have argued for years that the
Scholastic Assessment Tests are too
narrow a measure to determine who
should win admission to the country's best schools and get first
chance at the best jobs when they
graduate. Robert J. Sternberg, a professor of psychology at Yale University who is writing Michigan's new
tests, says the traditional tests are
"not going to tell you who has good
ideas."
"They're not going to tell you who
is going to have practical ideas," he
added. "They're not going to tell who
is going to get along with the boss."
In the mid-1980's, Dr. Sternberg
developed a concept he calls successful intelligence, or a combination of
analytical, creative and practical intelligence. Solving a math problem
or breaking down an essay requires
analytical intelligence. Figuring out
how to perform a task efficiently or
how to win over colleagues to a point
of view often involves the creative
and practical forms, he says.
Over the last two decades, Dr.
Sternberg has pointed to academic
studies to argue that many people
who display one form of intelligence
lack the others. The ability of a range
of workers, from garbage collectors
to horse race handicappers, to solve
problems as part of their jobs seems
to have little to do with their scores
on traditional standardized tests, according to the studies. Other psychologists disagree, saying that the connection between test scores and career success is significant.
Over all, though, "there is growing
concern that the tests we have been
using provide relatively weak indicators of longer-term success," said
Michael J. Feuer, the executive director of the Center for Education, a
division of the National Research
Council in Washington. In 1997, for
example, a study of doctors who had
graduated during a 20-year span
from the University of California at
Davis found no difference in career
success between students who had
been admitted with lower scores because of affirmative action and those
admitted because of their scores on
the medical school boards.
The debate over what intelligence
tests should and do measure is nearly as old as the exams themselves.
Alfred Binet, a French psychologist,
invented the original version in 1905.
As their use grew in the United
States in the following decades, researchers fiercely debated whether
they were putting too much of a premium on academic skills, according
to "The Big Test: The Secret History
of the American Meritocracy" by
Nicholas Lemann (Farrar Straus &
Giroux, 1999), a history of testing.
Corporate America has often split
the difference. Companies recruit
fast-track employees at elite universities that rely on standardized tests,
yet the companies tend to promote
workers for traits, like calmness under pressure and persuasiveness,
that do not show up on paper.
It is precisely these qualities that
Dr. Sternberg and Mr. White say a
written test can identify in a more
rigorous way than interviews, essays
or written recommendations can.
That idea is the most disputed part of
their effort.
Executives say that a valid test of
practical intelligence could be a
valuable tool in making hiring decisions, particularly because of the
speed with which managers must
now make decisions. "The biggest
gap in organizations today, from top
to bottom, is the ability to go from an
idea to a decision to getting it done,"
said John N. Fox, the vice chairman
of Deloitte Consulting and a member
of Michigan's advisory board.
The test, added Bruce W. Ferguson, the head of recruiting for the
consulting division at Ernst & Young,
could help companies figure out
which job a recruit is best suited for.
"It's not a silver bullet, but it becomes an additional data point," said
Mr. Ferguson, whose firm's consulting division hires 200 M.B.A.'s a year.
Even officials at the companies
that administer the existing standardized tests say they are excited by
the prospect of a test that tries to answer a different set of questions.
"I'm keenly interested to see what
they learn here," said Howard T.
Everson, the vice president for research at the College Board, which
oversees the Scholastic Assessment
Tests and other exams.
Other business school deans remain skeptical that the new test will
become a useful admission tool,
though. The rationale for the test is
that practical analysis is a skill that
students inherently have or lack,
said Donald P. Jacobs, the dean of
the business school at Northwestern
University. "I think I can teach you
how to be practical, to use theory to
come to practical conclusions," he
said.
In the end, education experts say,
Michigan's test will succeed or fail
based on its ability to make meaningful judgments about students. The
key question, Dr. Feuer of the National Research Council said, is
"What extent does this do a better
job of predicting anything than the
tests we currently have?"
