Hi A couple of observations to add to what others have said.
First, note that the reported p (.06 or .0619, precisely) is for a non-directional test. If the authors predicted the difference, directional p is half of this (.031) and significant. This is consistent with the moderate or large value of d, depending on your preferences there. Second, I think it could be risky to use a correlational effect size to estimate an experimental effect size. In principle it could go in either direction. Experimental effect size could be larger or smaller depending on whether correlated factors in correlational study contributed to or masked the observed effect. Also would have to infer whether the experimental manipulation was more or less "powerful" than naturally occurring variation on the predictor/independent variable. Third, even deciding in principle what effect size was required to be "important" is a challenging question. The labels of small, medium, and large are pretty meaningless without knowing what generalization is being made. In this case, for example, are we trying to generalize to the helping behavior of many millions of Americans, in which case a tiny effect size could be "important" (as in the classic aspirin study). In reality (versus theory), research design and statistics are messy and seemingly precise tools need to be used thoughtfully. Take care Jim Jim Clark Professor & Chair of Psychology 204-786-9757 4L41A From: Michael Britt [mailto:[email protected]] Also helpful. So, to answer my own previous question, based on what they found in the correlational study and what one might guess from previous research, I'm going to assume that the effect size here, if it exists, is probably small. So I used .3 in G*Power. The result? G*Power suggests that I get 242 subjects per group. These researchers had 26 subjects in each group. So: if you were the reviewer what would you conclude? The researchers found: "...the results revealed that participants in the anthropomorphism condition were tendentially less willing to help the victims of the natural disaster (M = 4.39, SD = 1.02) than participants in the control condition (M = 4.89, SD = 0.87), t(50) = -1.91, p = .06, d = 0.53. Would you recommend that they get more subjects? Michael Michael A. Britt, Ph.D. [email protected]<mailto:[email protected]> http://www.ThePsychFiles.com Twitter: @mbritt On Aug 27, 2013, at 8:59 AM, Stuart McKelvie <[email protected]<mailto:[email protected]>> wrote: Dear Tipsters, There are various ways to plan sample size. When teaching this in research methods, I divide the issues into two parts: 1. Estimation of population values. Here, more is better but there are diminishing returns. Think of the fact that we rarely see more than 1500 people in national polls and surveys. The formula is based on minimizing standard error. Of course, sampling is critical. 2. Conducting studies with variables: experimental, subject or correlational. There are four interconnected concepts: effect size, alpha, power and sample size. When any three are known, the fourth is determined. You can decide where to set alpha and power. For effect size (d), you can be guided by Cohen's guidelines for small, medium and large (.3, .5, .8) and choose the value you are looking for. This may come from past research or, in its absence, what you think is interesting theoretically or practically. Cohen's book on power analysis gives tables where you can look up the sample size needed after specifying the values you choose. There is also this webiste: http://homepage.stat.uiowa.edu/~rlenth/Power/ Sincerely, Stuart _____________________________________________________ Sent via Web Access "Floreat Labore" "Recti cultus pectora roborant" Stuart J. McKelvie, Ph.D., Phone: 819 822 9600 x 2402 Department of Psychology, Fax: 819 822 9661 Bishop's University, 2600 rue College, Sherbrooke, Québec J1M 1Z7, Canada. E-mail: [email protected]<mailto:[email protected]> (or [email protected]<mailto:[email protected]>) Bishop's University Psychology Department Web Page: http://www.ubishops.ca/ccc/div/soc/psy " Floreat Labore" _______________________________________________________ ________________________________ From: Paul C Bernhardt [[email protected]<mailto:[email protected]>] Sent: 27 August 2013 08:41 To: Teaching in the Psychological Sciences (TIPS) Subject: Re: [tips] Sample Size: How to Determine it? There is software to determine this. One excellent and free app is G*Power. http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/ I would use the correlational study to give me an estimate of effect size. As you describe, I would use that in the software to estimate my number of participants to attain the desired power. Practicality constraints on number of available participants usually limits things. I did such an estimate using G*Power a few weeks ago for a study we are planning. We will need to collect data over two semesters because the anticipated number of participants available from one semester's worth of students would only give us power of about .66, whereas two semester's worth would bump us up over .90. Paul On Aug 27, 2013, at 8:18 AM, Michael Britt wrote: I'm reading an interesting piece of research on anthropomorphism which essentially states after a natural disaster if we use the term "mother nature" when describing it, people will be less willing to contribute to relief efforts ("Humanizing nature could help the perceiver to conceive natural events as imbued with intentionality and significance rather than considering them merely random and meaningless phenomena"). They did two studies. Here's the issue/question: * Study 1 was correlational and involved 96 students. The results were supportive at <.001 * Study 2 was an experiment (no need to go into the details) involving 56 students. The results were, in the authors words, "tangentially" supportive with p<.06 I think the study was well conducted so I don't mean to slight the researchers. My guess is that if they used more subjects they probably would have reached p<.05 - but would that have been an example of "selective stopping"? I assume it would be. So how exactly does a researcher determine beforehand - as we are suggesting they do - the number of subjects they ought to try to get for the study? I'm just not familiar with the process. Does one look at the effect sizes of previous related studies to determine if the effect is large or small and then make a decision? But let's say the effect is assumed to be small, so do you use 100 subjects? 500? How is this number determined? Appreciate the insight in this. Michael Michael A. Britt, Ph.D. [email protected]<mailto:[email protected]> http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: [email protected]<mailto:[email protected]>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13441.4e79e96ebb5671bdb50111f18f263003&n=T&l=tips&o=27372 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27372-13441.4e79e96ebb5671bdb50111f18f263...@fsulist.frostburg.edu<mailto:leave-27372-13441.4e79e96ebb5671bdb50111f18f263...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: [email protected]<mailto:[email protected]>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13510.2cc18398df2e6692fffc29a610cb72e3&n=T&l=tips&o=27373 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27373-13510.2cc18398df2e6692fffc29a610cb7...@fsulist.frostburg.edu<mailto:leave-27373-13510.2cc18398df2e6692fffc29a610cb7...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: [email protected]<mailto:[email protected]>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13405.0125141592fa9ededc665c55d9958f69&n=T&l=tips&o=27374 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27374-13405.0125141592fa9ededc665c55d9958...@fsulist.frostburg.edu<mailto:leave-27374-13405.0125141592fa9ededc665c55d9958...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: [email protected]<mailto:[email protected]>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13251.645f86b5cec4da0a56ffea7a891720c9&n=T&l=tips&o=27379 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27379-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu<mailto:leave-27379-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=27381 or send a blank email to leave-27381-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
