Jay Warner wrote:
>
> Jay Tanzman wrote:
>
> > I just got chewed out by my boss for modelling the means of some 7-point
> > semantic differential scales. The scales were part of a written,
> > self-administered questionnaire, and were laid out like this:
> >
> > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful
> >
> > So, why or why not is it kosher to model the means of scales like this?
> >
> > -Jay
My boss's objection was that he believes "categorically" (sorry) that semantic
differential scales are ordinal.
> 1) Why do you think the scale is interval data, and not ordinal or
> categorical?
Why would anyone think it is ordinal and not interval? Most of the scales were
measuring abstract, subjective constructs, such as empathy and satisfaction, for
which there is no underlying physical or biological measurement. Why not, then,
_define_ degree of empathy as the subjects' rating on a 1-to-7 scale?
> If interval, the increments between the levels are more or
> less equal. If ordinal we know they are sequential, but have no idea how
> far apart each pair is. Categorical means there is no relationship between
> them - 4 is not greater than 3 - it's only different.
>
> Some people use a response of 4 to mean 'no response' as well as 'no
> opinion' and 'neutral opinion.' sorry, these are not intervals.
>
> 2) Is it possible for a respondent to come back with 2.5? If so, they
> think it is interval data, regardless of your opinion. Would you throw out
> a response of 2.5, or would you enter it in your dataset as 2.5? If the
> latter, you think it is interval, also.
An obscure corollary to the Law of Large Numbers is that, in a self-administered
questionnaire, the probability that some individual will either write in
some-number-point-five (or, equivalently, check two adjacent numbers) approaches
1 as N increases without bound. I would have no theoretical objection to them
doing that on this survey.
> 3) What makes you think the scale is linear (equal intervals)?
My boss's argument that it is not interval is that subjects don't necessarily
treat it that way. That is, they don't treat the difference between 1 and 2 as
the same as, say, between 3 and 4. My feeling is that there is no natural unit
of, say, satisfaction, so why not define a unit of satisfaction as the rating on
the scale.
> It ain't
> - since respondents can't go below 1 or above 7 . Well, maybe 0 and 8, but
> the point is the same. If you must, make a transformation (arc-sine for
> starters) to make it more 'linear' and more likely to contain Normal dist.
> data.
The scale can have limits and still be interval. The amount of water in an 8
oz. glass is constrained to be between 0 and 8, but ounces on water in the glass
would still be interval data.
> 4) Why might the respondents use the same increments that you think
> exist, or the same as other respondents? If there is some way you can
> 'anchor' at end points or mid point, you will get much more informative
> data. I mean, what is 'very stressful' to you? To me? to anyone?
I don't think it matters. What is 'very stressful' to the individual respondent
is what is important. For one thing, we were testing hypotheses about the
effects of alternative programs on these subjective outcomes. As long as there
was no association between how respondents interpret the scales and which
program they attended, I don't see how differences in scale interpretation could
affect the results; there would be no confounding.
> 5) In cases where I have been able to anchor firmly, and in some where I
> haven't, I find that treating the scale as incremental data work just fine,
> thank you.
I agree. Assuming that the data, which consist of the numbers 1 to 7, are
interval in the absence of evidence to the contrary seems like a pretty mild
assumption to me. Furthermore, even if they are not interval, treating them as
such would seem unlikely to cause any great bias in the results.
> As soon as you compute an average of responses on this scale,
> you have done just that. If you restrict yourself to categorical analysis
> for frequencies between categories, you have avoided that assumption. And
> you have far less to say about the data, as well.
Treating this data as categorical would have led to very sparse data. Ordinal
logistic regression would have been messy because I would had to collapse
categories, and this defeats the purpose of having the categories in the first
place. Treating the data as interval allowed me to evaluate the treatments and
their interactions using multiple linear regression, though, possibly, I could
have done this on the ranks of the data as well, though I didn't see any
advantage in doing so.
-Jay
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at
http://jse.stat.ncsu.edu/
=================================================================
