On Thu, 9 Dec 1999, Jim Clark wrote:
> HI
>
> On 7 Dec 1999, Magill, Brett wrote:
> > I am a graduate student in sociology studying individual's perceptions of
> > control (locus of control) using existing data. The data set include four
> > items to measure this construct which were taken from a larger scale of more
> > than twenty, the larger scale reaching an acceptable level of reliability (I
> > do not know the exact level, but it is a widely researched and used
> > instrument) in previous research. The four items that were included were
> > selected as the best measures of the construct based on empirical evidence
> > (item-total correlation's, factor analysis).
> >
> > In my own research, I used these items and decided to sum responses across
> > these four likert-type items. However, the Alpha reliability is very low
> > 0.30 (items were reverse scored as necessary and coding was double-checked).
> > I defended the decision to sum the items, despite the low Alpha, based on
> > the fact that they were selected from a larger set of items which are
> > internally consistent. In presenting my findings, I was heavily criticized
> > for this decision.
>
> One thing to point out to critics (and to those who argued
> against summating in response to your question) is that the
> reliability of a single-item is generally going to be lower than
> the reliability of a multi-item scale. I would determine the
> theoretically expected reliability of a 4-item scale, as
> suggested by another poster, and also do a few other things.
> Examine the Alpha output to see whether one of the items is
> causing particular problems. It may be that some set of 3 items
> has notably higher reliability. Do a factor analysis of the four
> items and see whether a single factor emerges and/or whether the
> pre-rotated loadings on the first factor are all positive.
>
> But you might also want to explore the possibility that indeed
> the 4 items do measure different things and have different
> relationships with your predictor variables. Can you easily
> repeat your analyses four times and do the results largely
> duplicate one another? If so, this would be another rationale
> for aggregating the items together and reporting a single
> analysis. Or if you factor analyze the 4 items along with your
> predictor variables, do the 4 items (or some subset) of them all
> load on the same factors?
>
Forgive me if I'm tuning in here at the end of the discussion, but I'm
wondering:
Who says locus of control is a pure, narrow-band concept? The Stanford
Binet intelligence test items yeild a low coefficient alpha. So do the
aggregated items from the Wechsler scales. But that's appropriate, if the
underlying dimension being measured is heterogeneous. cf Lee Cronbach's
discussion of bandwidth-fidelity in his books on tests & measures.
Doesn't it also make a difference what the original item pool was that was
factored? What scale are we talking about?
If the population being tested is different from the one the factor
analysis was based on, it might account for the drop in coef. alpha.
Don't you have to be concerned with why the drop? (I'm assuming in the
original scale the coef. alpha was relatively high, even if, in the
original group, you only looked at these four items.)
Mike
