There is no bending of the assumptions because incorrect/correct is not
categorical. Not all dichotomies are nominal and categorical. For
example, if I dichotomized income into below 50K (0) and greater than
50K (1), it doesn't make income nominal. This operation removes
information from the scale but it is
still an interval scale. The conventional way to analyze the data in
the study below is to just compare pre/post on each item using a t
test. If ANOVA was used in the
way proposed, you would still need to conduct t tests in order to find
which specific items were significantly different. For effect size
measurement, just use Cohen's d.
I think the general issue here is the conceptualization of dichotomies.
Even categorical variables can all be dummy coded into dichotomies
and analyzed using the general linear model.
Mike Williams
Subject: RE: Re:my crummy knowledge of stats
From: Paul C Bernhardt <[email protected]>
Date: Thu, 17 Jan 2013 12:39:15 +0000
X-Message-Number: 2
If you want to bend the assumptions of statistics to this degree (and
I'm not saying it is wrong, because there is plenty of evidence to
support the robustness of t-test and ANOVA to all kinds of violations of
assumptions, though I'm not sure about this particular choice, I'd want
validating study to back me up), why not go for two-way within-subjects
ANOVA? One variable is pre-post and the other is question number.
Paul
________________________________________
From: Mike Wiliams [[email protected]]
Sent: Thursday, January 17, 2013 12:19 AM
To: Teaching in the Psychological Sciences (TIPS)
Subject: Re:[tips] my crummy knowledge of stats
You can use a conventional paired t test. Although you have dichotomous
scores that does not mean they are categorical. Correct/incorrect is a
ratio scale of 1 unit.
Green/Red, Accountant/Psychologist are the type of categorical
dichotomies that bring in the nonparametric procedures like Chi-square
or ranking tests.
Just calculate a mean difference and variance for each item and analyze
them the usual way. You might also try some of the test reliability
stats that are now in
SPSS, such as coefficient alpha. Alpha is a general index of how well
the items intercorrelate or "hang together".
Mike Williams
----- Original Message -----
From: "Annette Taylor"<[email protected]>
To: "Teaching in the Psychological Sciences
(TIPS)"<[email protected]>
Sent: Tuesday, January 15, 2013 6:21:42 PM
Subject: [tips] my crummy knowledge of stats
I know this is a basic question but here goes:
I have categorical data, 0,1 which stands for incorrect (0) or correct
(1) on a test item.
I have 25 items and I have a pretest and a posttest and I want to know
on which items students improved significantly, and not just by chance.
Just eyeballing the data I can tell that there are some on which the
improved quite a bit, some not at all and some are someplace in the
middle and I can't make a guess at all. That is why we have statistics.
Yeah! .... hmmmm....bleh.....
As far as I know, the best thing to do is a chi-square test for each of
25 items; but of course that will mean that with a .05 sig level I will
have at least one false positive, maybe more, but most assuredly at
least one. This seems to be a risk. At any rate I can use SPSS and the
crosstabs command allow for calculation of the chi-square.
I know that when I do planned comparisons with multiple t-tests, I can
do a Simes' correction in which I can rank order my final, obtained
alphas, and adjust for the number of comparisons and reject from the
point from which the obtained alpha failed to exceed the
corrected-for-number-of-comps alpha. But as far as I know, I cannot do
that with 25 chi square tests. There is probably some reason why I can
no more do that, that relates to the reason for why I cannot do 25
t-tests in this situation with categorical data.
Is there a better way to answer my research question? I need a major
professor! Oh wait, that's me... drat! I need to hire a statistician. Oh
wait, I'd need $$ for that and I don't have any. So I hope tipsters can
stand in as a quasi-hired-statistician and help me out.
Oh, I get the digest. I don't mind waiting until tomorrow or the next
day for a response, but a backchannel is [email protected]
I will be at APS this year. Any other tipsters planning to be there?
Let's have a party! I'd love to put personalities to names.
Thanks
Annette
Annette Kujawski Taylor, Ph. D.
Professor, Psychological Sciences
University of San Diego
5998 Alcala Park
San Diego, CA 92110
[email protected]
---
You are currently subscribed to tips as: [email protected].
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=23136
or send a blank email to
leave-23136-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
