Hi Annette Perhaps McNemar's Test for significance of changes, for dichotomous data. For each item, set up a table that looks like a 2*2 chi square but has "pretest" and "post-test" as variables (in texts its usually labelled "before" and "after") . Posttest - + + A B Pretest - C D
So every S appears as one count in the table and A+B+C+D = total N. But the key cells are A and D. Since (A+D) equals the people who changed, we expect half of (A+D) in A and D if the Null is true. The frequency expected for both A and D is (A+D)/2 ... Then you just do a chi square on those two cells. The formula simplifies to chi square = (A - D)squared / (A + D) with df = 1. The Wikipedia link is http://en.wikipedia.org/wiki/McNemar%27s_test, and I just noticed it reorders the rows and columns so that B and C are the "change" variables. It also reminds us about Yates correction .. Unfortunately, I don't use SPSS much these days so I don't know how to find it or code the variables for the chi square! Not sure about the chance issue. The p values may be all you need ,.. but you may want a correction for chance?? Others will know more .... ========================== John W. Kulig, Ph.D. Professor of Psychology Coordinator, University Honors Plymouth State University Plymouth NH 03264 ========================== ----- Original Message ----- From: "Annette Taylor" <[email protected]> To: "Teaching in the Psychological Sciences (TIPS)" <[email protected]> Sent: Tuesday, January 15, 2013 6:21:42 PM Subject: [tips] my crummy knowledge of stats I know this is a basic question but here goes: I have categorical data, 0,1 which stands for incorrect (0) or correct (1) on a test item. I have 25 items and I have a pretest and a posttest and I want to know on which items students improved significantly, and not just by chance. Just eyeballing the data I can tell that there are some on which the improved quite a bit, some not at all and some are someplace in the middle and I can't make a guess at all. That is why we have statistics. Yeah! .... hmmmm....bleh..... As far as I know, the best thing to do is a chi-square test for each of 25 items; but of course that will mean that with a .05 sig level I will have at least one false positive, maybe more, but most assuredly at least one. This seems to be a risk. At any rate I can use SPSS and the crosstabs command allow for calculation of the chi-square. I know that when I do planned comparisons with multiple t-tests, I can do a Simes' correction in which I can rank order my final, obtained alphas, and adjust for the number of comparisons and reject from the point from which the obtained alpha failed to exceed the corrected-for-number-of-comps alpha. But as far as I know, I cannot do that with 25 chi square tests. There is probably some reason why I can no more do that, that relates to the reason for why I cannot do 25 t-tests in this situation with categorical data. Is there a better way to answer my research question? I need a major professor! Oh wait, that's me... drat! I need to hire a statistician. Oh wait, I'd need $$ for that and I don't have any. So I hope tipsters can stand in as a quasi-hired-statistician and help me out. Oh, I get the digest. I don't mind waiting until tomorrow or the next day for a response, but a backchannel is fine. [email protected] I will be at APS this year. Any other tipsters planning to be there? Let's have a party! I'd love to put personalities to names. Thanks Annette Annette Kujawski Taylor, Ph. D. Professor, Psychological Sciences University of San Diego 5998 Alcala Park San Diego, CA 92110 [email protected] --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13338.f659d005276678c0696b7f6beda66454&n=T&l=tips&o=23044 or send a blank email to leave-23044-13338.f659d005276678c0696b7f6beda66...@fsulist.frostburg.edu --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=23065 or send a blank email to leave-23065-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
