First, forget the responses from Chen and Donovan. No laws are being
violated by adding these items, or any items, together. And traditional
psychometric methods of evaluating the "unidimensionality" of items are not
going to be very persuasive with only 4 items.
As PChawla pointed out, alpha is a function both of the number of items in
the list and intercorrelations among them. If the intercorrelations of the 4
items are what you expected them to be from the literature you should be able to
call upon that literature to support your decision to use them as a summed
scale. Why on earth would anyone prefer to use the items individually if face
validity and previous usage supports the notion that they are all measuring what
you say they are?
Consider this: if I have a math test made up of 20 questions, and everybody
agrees that adding up the number right to get a total score is a good way to
decide on a student's grade, wouldn't you also add up the number right and use
if only 5 of the questions happened to be available? How reliably the total
score measures the concept needs to be separated from whether the total score is
a respectable measure of the concept. The VALIDITY of the shorter test should be
distinguished from its (obviously lower) RELIABILITY.
You say you are a sociologist, but your critics are acting like
psychologists in their apparent concern with Alpha reliability. . Have you
considered it possible that their disappointment with your study might be due to
the fact that "locus of control" is an outdated concept that has been overused
by graduate students for 4 decades without contributing much of anything to our
understanding of social behavior?
It may not help you against your critics, but I suggest that you look at
chapter 9 of Dave Caplovitz's 1983 book "The Stages of Social Research" (and
some of the references cited therein) for a classic discussion of how survey
researchers view the issue of "Concepts and indices: the process of
measurement".
Neil Henry, Virginia Commonwealth University
"Magill, Brett" wrote:
> Just wanted people's thought on the following:
>
> I am a graduate student in sociology studying individual's perceptions of
> control (locus of control) using existing data. The data set include four
> items to measure this construct which were taken from a larger scale of more
> than twenty, the larger scale reaching an acceptable level of reliability (I
> do not know the exact level, but it is a widely researched and used
> instrument) in previous research. The four items that were included were
> selected as the best measures of the construct based on empirical evidence
> (item-total correlation's, factor analysis).
>
> In my own research, I used these items and decided to sum responses across
> these four likert-type items. However, the Alpha reliability is very low
> 0.30 (items were reverse scored as necessary and coding was double-checked).
> I defended the decision to sum the items, despite the low Alpha, based on
> the fact that they were selected from a larger set of items which are
> internally consistent. In presenting my findings, I was heavily criticized
> for this decision.
>
> Now, I could use individual items and a procedure such as logistic
> regression (I was using GLM before with this scale as the dependent and a
> sample of better than 5000) without changing my conclusions (I ran logistic
> models anticipating the criticism), however I was not convinced that this is
> necessary.
>
> My question is, is summing these items defensible or at least as defensible
> as summing any set of likert-type items to produce a single score. Where
> could I find support for what I am doing if it is (clearly my peers won't
> just take my word for it)?
>
> Regards,
>
> Brett
--
Neil
