Don Allen wrote:
> Fellow Tipsters-
>
> I need a bit of help from the more experienced stat mavens on this
> forum. A student of mine has been conducting some survey research and is
> now having difficulty in analysing the data. Part of her survey
> consists of nominal data (e.g. have you ever been the victim of a bully?
> Y/N) which was asked both pre & post. She is attempting to do a Chi
> Square analysis to see if the response rates changed over time. The
> problem is that (as usual) not all subjects completed the second
> questionnaire. Since she has a pretty good sample size (N= 135) she
> could simply discard all of the data from Ss who did not complete both
> parts. However, this is a three year cross-sequential design and we are
> afraid that this approach would leave us with a greatly reduced N. The
> problem, then, is how to calculate the Expected values given the unequal
> Ns. I thought that a ratio comparison would be a reasonable way to
> approach this (e.g. if 85/135 said "Y" on the pretest & 79/126 said "Y"
> on the posttest then you could use the first value as the Expected
> values & the second value as the Observed. A local stat expert has told
> me that this is not allowed but he wasn't clear on why. So the question
> is: can you use this ratio method to compute Chi Square or, if not, what
> other method could you use that would help to preserve N?
The McNemar test should work, but you have to discard those Ss who did not
complete the second part. Set it up as:
(-) After (+) After
(+) Before A B
(-) Before C D
We're interested only in cells A and D (if I read the scenario correctly) -
i.e. whether more people switch from + to - vs. from - to +. Since A and D
are the only focus in this example, the computational formula for chi-square
= (A - D)^2 / (A + D), df = 1. Note that this preserves the independence of
cells (A person falls into one, and only one of the 4 categories). If set up
correct, N (the number of actual people you used) = A+B+C+D. A common
(incorrect) way to set up the table is:
(-) (+)
Before A B
After C D
This doesn't work because the cells now do not have independent data and a
person falls in two categories (or in this case, some people fall in 2
cells, and others fall in one). And, A+B+C+D <> N here. Siegel's
_NonParametric Statistics_ has more information on the correct (McNemar)
test. I think you're stuck discarding those that didn't do both measures.
The above assumes the student is interested in whether the tendency to shift
answers from - to + differs from the tendency to shift from + to -. That is
the point of the McNemar test. But the student may just be interested in the
data in a simplier and more descriptive way - e.g. 75% of people said "yes"
at this point but 85% said "yes" at later point. That can be useful data to
know, but just report it descriptively and don't do inferential stats. There
are simple tests for differences in proportions ( z = (proportion 1 -
proportion 2)/standard error) but you wouldn't want to do that because the
proportions are not independent (i.e. some people contributed data to both
proportions).
>
> A second question relates to the the large number of comparisons that
> she is attempting to do (N > 30). My first reaction was to tell her to
> do a Bonferoni correction, but that tends to be a bit conservative
> (maybe not a bad thing). Is there another correction for non-parametric
> stats that would be more appropriate?
I'm not supportive of Bonferroni. I'd rather see the student just adopt a
slightly higher hurdle (e.g. .01), and/or par down the number of comparisons
that are the main focus of the research. I hope this helps!
--
---------------------------------------------------------------
John W. Kulig [EMAIL PROTECTED]
Department of Psychology http://oz.plymouth.edu/~kulig
Plymouth State College tel: (603) 535-2468
Plymouth NH USA 03264 fax: (603) 535-2412
---------------------------------------------------------------
"What a man often sees he does not wonder at, although he knows
not why it happens; if something occurs which he has not seen before,
he thinks it is a marvel" - Cicero.
