[EMAIL PROTECTED] wrote: > I really do appreciate the time you folk took to respond, but I am > still in the dark. So let me be a bit more explicit and see if what I > need makes more sense. > > I am a sixth grade teacher. I have two sets of reading scores > (standard scores with a range of 27 to 55). One set has 40 students > and the other has 65. I enter each score in a column in Excel for > each class. I get the mean, st. dev. and variance from Descriptive > Statistics. On one test the difference in mean scores is less than 2 > points. On another test the difference is 12 points. Common sense > tells me there is a significant difference in the two sets of scores > on the test with 12 points difference. My question is: is the > difference of 2 points statistically significant or the result of > chance? If my assumption is correct and one class did significantly > better than the other on one test and on the other the difference in > performance is (probably) not statistically significantly different > then is that a correct assumption? So somewhere between 12 points and > 2 points I can guess there is a significant difference but less than > that it is not. I asked a math teacher how to determine that point on > any given set of scores and was told to run a z test. I was told a t > test is used to test whether a sample's mean is representative of the > total population's mean. So I used a z test. In Excel what is > returned is a "z" score, a P score for one tail and a P score for two > tail for a hypothesized mean of .05. Can someone tell me how to > interpret the numbers returned for z, P one tail and P two tail, > assuming the z test is the correct procedure. If not, what do I use > and how do I interpret the results Excel returns?
First, what are your goals? If you merely want to reach the conclusion that one set of scores is better than the others you need no test. Similarly, if you want to conclude that 12 points is a bigger difference than 2 points you need no test. You might want to standardize the differences in terms of the standard deviations of the tests e.g., 2/x and 12/y to make them more comparable if the s.d.s of the tests are different. When do you need a significance test? When you want to generalize these sets of data to a larger population of possible data sets. For example, to generalize to similar students, to the same students at a future date it might be reasonable to use a significance test as one part of the procedure. If so, I'd use a t test. This allows us to generalize to a hypothetical population of similar data sets (it is a misconception to think that a specific population - such all students or all students in Kentucky, or all students wearing blazers - is being generalized to). The t test (or any other test) won't account for bias in sampling (so if the students you test aren't similar to the ones you want to generalize to you are stuck). How to interpret? The test is significant if the observed p (two-tailed) falls below a certain threshold called alpha (typically alpha is set at 0.05 or 5%). This p is associated with a t value a certain size with a certain sample size (as indicated by the d.f.). A significant result means you have evidence that the population means are different. For example, if you were to collect two similar data sets in future you'd expect them to show a difference in the same direction. A non-significant result means you have insufficent evidence either way as the results could have been produced by purely random process (though we can never be sure they were). This is a very, very basic summary of the main points (I've omitted a detailed discussion of null hypothesis testing, power and so forth). Most important I've left out consideration of lots of other things that influence the interpretation of the test. These include the way the data were collected, potentiall sources of bias and so forth. Thom . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
