Bryan Caplan wrote:
> Most of these wind up being dependent variables at some point in their
> book.
I'm not sure why that helps their case at all -- it's as if they've
produced a whole bunch of reduced form equations, treating age,
and, probably inappropriately, SES and AFQT as the only exogenous
regressors. Take any one of the outcomes they consider and the
relevant related discussion. Cast it as a study on that particular
outcome. Would it be accepted by any economics journal? I don't
think so -- so why should I think that if we put a couple of dozen
of these together, we arrive at something compelling?
> variable. I think there are good practical reasons not to do this, and
> there is a wealth of research that uses simplified indices in place of
> the kitchen sink (e.g. "Democracy" indices, "Rule of Law" indices, "Bank
> Failure" indices, etc.)
Sure, but here there was no reason not to simply put the four variables
in each regression, rather then the ad hoc index.
> My memory is not too good here - I read a few pieces by Heckman on this,
> but nothing that I remember reaching results that were "dramatically"
> different.
You don't take it as problematic for M/H's conclusions that Heckman
found there was no way to seperate the effects of intelligence from
education? That he and coathors showed the returns to intelligence
vary markedly across subpopulations?
> What you call an "endogeneity problem" doesn't fit the usual textbook
> description. Normally correlation among independent variables isn't a
> problem, though it complicates the interpretation if changing one
> variable almost always changes the other.
I am referring to correlation between the error term and regressors as
"endogeneity," which is of course the usual textbook defintion.
Correlation amongst the independent variables is actually a problem (it
reduces precision of the estimates), it's just not a problem that causes
inconsistency. I suggest, here, that omitted variables, correlation
between unobservables, measurement error, and reverse causation are all
contributing to profound endogeneity problems.
> Normally measurement error just yields attenuation bias - making all
> coefficients on the poorly measure variables too small in absolute
> value. But as best I could tell you got your result with correlated
> measurement error across X variables. Is that right?
Yes. But keep in mind measurement error _is_ a source of endogeneity --
that's actually the point, in the context of labor market outcomes and
IQ, of the first half of that AER piece I mentioned.
> > That said, I'm not sure education, at least, is particularly
> > well measured in most datasets, as we generally ignore quality measures.
>
> Well-measured compared to what?
Lots of things (to take some obvious examples: gender, age, nationality,
region of residence). Surely there's an important difference between the
four years of education one gets at the lowest-ranked religious teaching
college and a four-year degree from MIT, no? Of course, this would be a
more interesting point if M/H had actually controlled for years of
education.
> > And recall M/H "solve" the colinearity problem they have between education
> > and IQ by *dropping* education. Some solution!
>
> Though it may appall you, many textbooks recommend it.
Many textbooks recommend dropping a regressor, then interpreting the
coefficients on the remaining regressors _as if_ the dropped regressor
was still in the equation? That's clearly not true. What are these
"many textbooks," out of curiosity (I can't recall ever reading anyone
recommending that one "solve" colinearity problems by just tossing
out regressors)? And surely they note that such an exclusion affects how
one should interpret the coefficients on the remaining regressors, no?
> In spite of all
> of his reservations about TBC, Bill Dickens felt pretty comfortable with
> their omission.
Perhaps Bill can explain why?
> I'd put it differently. Leaving education out clearly understates its
> importance. But putting in both IQ and education can also give a
> misleading interpretation insofar as its hard for one to change without
> changing the other. In other words, you're choosing between two
> imperfect strategies, and its not obvious to me that M/H chose wrongly.
The point is that given their imperfect strategy, they should have
realized that the coefficient on AFQT isn't the impact of cognitive
ability, all else equal. It's obviously impossible to tell in their
regressions whether sheer cognitive ability has any direct effect beyond
education, income, and all the other omitted variables. If they'd said
that, their discussion wouldn't be so grating, although then one would
wonder why they bothered with nearly meaningless regression after nearly
meaningless regression.
> If anything your command of the details of their study exceeds mine.
> The main thing I'd say in their defense is to step back and take stock
> of what they did. Here's how I'd summarize it:
>
> 1. They put forward an intutively plausible hypothesis that virtually
> no economists I knew at Berkeley or Princeton ever even mentioned.
I agree that cognitive ability is a too-often omitted variable. But it
isn't the case that it hasn't appeared in the economics literature before
(see, for instance, that AER piece I've mentioned, which is more plausible
than anything in TBC).
> 2. They showed that this hypothesis explained an extremely diverse and
> seemingly unconnected sets of facts.
They didn't even come within the ballpark of holding all else equal, so
I disagree they "showed" this result at all. Consider all their outcomes
but poverty, and suppose we live in a world where cognitive ability has
zero impact on whatever the outcome of interest is, but income has a large
impact of the same sign as M/H find with respect to AFQT. How would we be
able to discern whether we live in such a world from the regressions in
TBC?
> 3. They directly challenged almost the entire "socialization"
> literature within psych, and implicitly called much of the ROR to
> education literature in econ into question.
I'm not sure why. It's long been extremely well-known that education is
an endogenous outcome in wage equations. I don't recall M/H ever actually
running an income wage equation (I could be wrong, don't have the book
handy at the moment), but they certainly woulnd't have been the first to
stick proxies for ability in to try to trick out the direct effect of
education.
> 4. Almost all of the econometric problems you cite seem at least as
> severe in almost every study I can think of. Correlated independent
> variables, measurement error, using indices, etc. If these were enough
> for summary rejection, most of the ROR to education literature would
> remain unpublished.
It's not that these problems exist in TBC per se that leads to me the
conclusion it's all but worthless, it's the sheer magnitude of these
problems, combined with the analysis that seems blithely unaware that
these problems undercut their conclusions. All they're holding constant
in most specifications is the SES index and age. All else aside,
endogeneity via the dozens of surely relevant but omitted variables is
enough to render their results meaningless.
When was the last time you saw an economist run a wage equation and
include only age and an ad hoc index of parents' socioeconomic status
constant, in addition to some variable of particular interest in the
study? How about a determinants of criminal activity equation that
doesn't control for _gender_?
> Of course, saying "*I* learned a lot from it" might just be a sign of
> how clueness I was or am. But I'm pretty sure that even today most
> labor economists are happily ignoring the issues M/H raise. In short,
> most economists would learn a lot from TBC. How many published articles
> can you say that about?
I think we're talking across each other to some extent here. I'm not
saying TBC doesn't discuss lots of interesting questions (although I don't
think most of those questions are attributable to M/H). I'm certainly
not saying I didn't learn a lot from it (although that view is tempered
somewhat by oft-made claims that M/H's review of the literature is
biased). What I'm saying is that the original statistical results in
the TBC are so rotten that they can't be used to show much of anything,
and they certainly don't show what M/H think they show. This doesn't
necessarily imply that M/H are wrong in their qualitative conclusions,
of course. But we will have to look elsewhere to find if their results
hold under the much more rigorous analysis required to get a solid
handle on these question.
Chris Auld (403)220-4098
Economics, University of Calgary <mailto:[EMAIL PROTECTED]>
Calgary, Alberta, Canada <URL:http://jerry.ss.ucalgary.ca/>
