Re: [tips] What to do with skewed data
On Mon, 11 Nov 2013, Michael Britt went: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person So you've got yourself the mirror image of one of these: http://psycnet.apa.org/psycinfo/1983-09462-001 |The insidious L-shaped distribution. |Bradley, James V. |Bulletin of the Psychonomic Society, Vol 20(2), Aug 1982, 85-88. | |L-shaped distributions are probably more prevalent than generally |realized. They are highly conducive to nonrobustness of |normality-assuming statistical tests, and strongly resist |transformation to normality. The thinner the tail of the |distribution, the more unlikely it is that its L-shapedness will be |detected by inspecting a sample drawn from it. Yet, as the tail of an |L-shaped distribution becomes increasingly shallow, its skewness and |kurtosis depart increasingly from their normal-distribution values, |and the distribution becomes increasingly conducive to drastic |nonrobustness. Worse, a fairly common type of experimental situation |in psychological research produces shallow-tailed L-shaped |distributions. If you do a search on statistical techniques for zero-inflated continuous [or semicontinuous] data, you might be able to apply them to your ten-inflated data. --David Epstein da...@neverdave.com --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30025 or send a blank email to leave-30025-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
Re: [tips] What to do with skewed data
On 11/11/2013 1:01 PM, Michael Britt wrote: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person (or there's some form of cognitive dissonance going on, but that's not my question. No matter how big the sample size (mine was 160 respondents) I assume you'll always get a skewed distribution on a question like this so wouldn't I be breaking the normalization assumption if I were to do correlations using these results? I assume I could either do: a) do some kind of transformation - but I've never done one before so I’m not familiar with it, or b) recode the data into 3 categories (perhaps 1-5 is low satisfaction, 6-7 is moderate and 8-10 is high) and do a chi-squre instead of a correlation. Any thoughts? Appreciate it. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt Hi Michael: Tabachnick and Fidell (2007, 5th ed., pp. 86-90) suggest the following transformation for negatively-skewed data. First, you should create a new variable that is a reflection of the original data: NewX = K - OldX ;where K = your max score + 1. Then try a square root or a log transform of NewX and see which transform produces the smallest skew score. Finally, remember that the reflection reverses the direction of your scale. A 10 indicates low satisfaction. Ken Kenneth M. Steele, Ph. D.steel...@appstate.edu Professor Department of Psychology http://www.psych.appstate.edu Appalachian State University Boone, NC 28608 USA --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30026 or send a blank email to leave-30026-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
RE: [tips] What to do with skewed data
Dear Tipsters, Although the Pearson coefficient does not assume normality, an alternative solution might be to computer a non-parametric coefficient such as Spearman's rho or Kendall's tau. Sincerely, Stuart __ Recti Cultus Pectora Roborant Stuart J. McKelvie, Ph.D., Department of Psychology, Bishop's University, 2600 rue College, Sherbrooke (Borough of Lennoxville), QC J1M 1Z7, Canada. Floreat Labore __ --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30027 or send a blank email to leave-30027-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
Re: [tips] What to do with skewed data
You don't mention the other variables to which this is to be related. If you planned to do a correlation between this variable and another interval variable, a Spearman rank order could be used. If the other variable is a nominal grouping variable, another non parametric procedure could be used, depending on the number of groups. Rick Rick Froman rfro...@jbu.edu On Nov 11, 2013, at 12:02 PM, Michael Britt mich...@thepsychfiles.com wrote: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person (or there's some form of cognitive dissonance going on, but that's not my question. No matter how big the sample size (mine was 160 respondents) I assume you'll always get a skewed distribution on a question like this so wouldn't I be breaking the normalization assumption if I were to do correlations using these results? I assume I could either do: a) do some kind of transformation - but I've never done one before so I’m not familiar with it, or b) recode the data into 3 categories (perhaps 1-5 is low satisfaction, 6-7 is moderate and 8-10 is high) and do a chi-squre instead of a correlation. Any thoughts? Appreciate it. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: rfro...@jbu.edu. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13039.37a56d458b5e856d05bcfb3322db5f8an=Tl=tipso=30023 or send a blank email to leave-30023-13039.37a56d458b5e856d05bcfb3322db5...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30028 or send a blank email to leave-30028-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
RE: [tips] What to do with skewed data
As soon as you do anything involving t or F (like test a hypothesis or construct a confidence interval), there is a normality assumption. As Stuart notes, computing the r requires no distribution assumptions. Of course the same is true of the mean. Then again, r is really just a special mean. Cheers, Karl W. -Original Message- From: Stuart McKelvie [mailto:smcke...@ubishops.ca] Sent: Monday, November 11, 2013 1:47 PM To: Teaching in the Psychological Sciences (TIPS) Subject: RE: [tips] What to do with skewed data Dear Tipsters, Although the Pearson coefficient does not assume normality, an alternative solution might be to computer a non-parametric coefficient such as Spearman's rho or Kendall's tau. Sincerely, Stuart __ Recti Cultus Pectora Roborant Stuart J. McKelvie, Ph.D., Department of Psychology, Bishop's University, 2600 rue College, Sherbrooke (Borough of Lennoxville), QC J1M 1Z7, Canada. Floreat Labore __ --- You are currently subscribed to tips as: wuens...@ecu.edu. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13060.c78b93d4d09ef6235e9d494b3534420en=Tl=tipso=30027 or send a blank email to leave-30027-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30031 or send a blank email to leave-30031-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
Re: [tips] What to do with skewed data
Thanks for that link to the article on L distributions. I don't think I quite got an L. Here's what the data looked like: http://i.imgur.com/HtOCRc1.jpg But still, I'll take a look at the article since it addresses what to do with non-normal distributions. Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt On Nov 11, 2013, at 1:29 PM, David Epstein da...@neverdave.com wrote: On Mon, 11 Nov 2013, Michael Britt went: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person So you've got yourself the mirror image of one of these: http://psycnet.apa.org/psycinfo/1983-09462-001 |The insidious L-shaped distribution. |Bradley, James V. |Bulletin of the Psychonomic Society, Vol 20(2), Aug 1982, 85-88. | |L-shaped distributions are probably more prevalent than generally |realized. They are highly conducive to nonrobustness of |normality-assuming statistical tests, and strongly resist |transformation to normality. The thinner the tail of the |distribution, the more unlikely it is that its L-shapedness will be |detected by inspecting a sample drawn from it. Yet, as the tail of an |L-shaped distribution becomes increasingly shallow, its skewness and |kurtosis depart increasingly from their normal-distribution values, |and the distribution becomes increasingly conducive to drastic |nonrobustness. Worse, a fairly common type of experimental situation |in psychological research produces shallow-tailed L-shaped |distributions. If you do a search on statistical techniques for zero-inflated continuous [or semicontinuous] data, you might be able to apply them to your ten-inflated data. --David Epstein da...@neverdave.com --- You are currently subscribed to tips as: michael.br...@thepsychfiles.com. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13405.0125141592fa9ededc665c55d9958f69n=Tl=tipso=30025 or send a blank email to leave-30025-13405.0125141592fa9ededc665c55d9958...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30032 or send a blank email to leave-30032-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
RE: [tips] What to do with skewed data
Classic solution: use a rank transformation -- that is, use the Spearman coefficient rather than the Pearson coefficient. Alternative -- use Kendall tau Another alternative -- do as Ken suggests, but need not reflect first, just use square instead of square root (you can alter the strength of this transformation by changing the exponent) or use exponential transformation instead of log transformation. Modern solution -- bootstrap it. David Howell has, on his website, free resampling software for simple designs. Cheers, Karl W. -Original Message- From: Michael Britt [mailto:mich...@thepsychfiles.com] Sent: Monday, November 11, 2013 1:01 PM To: Teaching in the Psychological Sciences (TIPS) Subject: [tips] What to do with skewed data I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person (or there's some form of cognitive dissonance going on, but that's not my question. No matter how big the sample size (mine was 160 respondents) I assume you'll always get a skewed distribution on a question like this so wouldn't I be breaking the normalization assumption if I were to do correlations using these results? I assume I could either do: a) do some kind of transformation - but I've never done one before so I'm not familiar with it, or b) recode the data into 3 categories (perhaps 1-5 is low satisfaction, 6-7 is moderate and 8-10 is high) and do a chi-squre instead of a correlation. Any thoughts? Appreciate it. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: wuens...@ecu.edu. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13060.c78b93d4d09ef6235e9d494b3534420en=Tl=tipso=30023 or send a blank email to leave-30023-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30030 or send a blank email to leave-30030-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
Re: [tips] What to do with skewed data
Michael, There are couple of standard ways to transform skewed data. Invert the data (subtract each datum from one greater than the highest value) so that the skew is positive. Then, Depending of the strength of the skew, do a square root or logarithmic transformation. Alternatively, don't invert it and take the reciprocal (1/x) of each datum). Chris ... Christopher D Green Department of Psychology York University Toronto, ON M6C 1G4 chri...@yorku.ca http://www.yorku.ca/christo On Nov 11, 2013, at 1:01 PM, Michael Britt mich...@thepsychfiles.com wrote: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person (or there's some form of cognitive dissonance going on, but that's not my question. No matter how big the sample size (mine was 160 respondents) I assume you'll always get a skewed distribution on a question like this so wouldn't I be breaking the normalization assumption if I were to do correlations using these results? I assume I could either do: a) do some kind of transformation - but I've never done one before so I’m not familiar with it, or b) recode the data into 3 categories (perhaps 1-5 is low satisfaction, 6-7 is moderate and 8-10 is high) and do a chi-squre instead of a correlation. Any thoughts? Appreciate it. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: chri...@yorku.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=430248.781165b5ef80a3cd2b14721caf62bd92n=Tl=tipso=30023 or send a blank email to leave-30023-430248.781165b5ef80a3cd2b14721caf62b...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30040 or send a blank email to leave-30040-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
Re: [tips] What to do with skewed data
Never use Spearman's rho (which idiotically assumes a rectangular distribution). Always use Kemdall's tau ( which makes no distributional assumption and has a much more sensible interpretation). That is all. :-) Chris ... Christopher D Green Department of Psychology York University Toronto, ON M6C 1G4 chri...@yorku.ca http://www.yorku.ca/christo On Nov 11, 2013, at 1:46 PM, Stuart McKelvie smcke...@ubishops.ca wrote: Dear Tipsters, Although the Pearson coefficient does not assume normality, an alternative solution might be to computer a non-parametric coefficient such as Spearman's rho or Kendall's tau. Sincerely, Stuart __ Recti Cultus Pectora Roborant Stuart J. McKelvie, Ph.D., Department of Psychology, Bishop's University, 2600 rue College, Sherbrooke (Borough of Lennoxville), QC J1M 1Z7, Canada. Floreat Labore __ --- You are currently subscribed to tips as: chri...@yorku.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=430248.781165b5ef80a3cd2b14721caf62bd92n=Tl=tipso=30027 or send a blank email to leave-30027-430248.781165b5ef80a3cd2b14721caf62b...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30041 or send a blank email to leave-30041-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
RE: [tips] What to do with skewed data
Hi People have been noting various transformations, depending on reversing or not the original. In essence many common transformations can be conceptualized as the original scores raised to various powers, with the powers being greater than or less than 1. x^-1 = reciprocal 1/x x^~0 = logarithmic x^.5 = square root x^1 = original x^2 = ... I'm mostly used to thinking of these in terms of various non-linear relationships (powers 1 compress upper end, powers 1 expand upper end), but some of these will increase skewness and others will decrease it, depending on the direction of skew. Possible to experiment with them to observe effect. Perhaps also worth plotting some of the relationships you are interested in? Take care Jim Jim Clark Professor Chair of Psychology 204-786-9757 4L41A -Original Message- From: Christopher Green [mailto:chri...@yorku.ca] Sent: Monday, November 11, 2013 5:31 PM To: Teaching in the Psychological Sciences (TIPS) Subject: Re: [tips] What to do with skewed data Michael, There are couple of standard ways to transform skewed data. Invert the data (subtract each datum from one greater than the highest value) so that the skew is positive. Then, Depending of the strength of the skew, do a square root or logarithmic transformation. Alternatively, don't invert it and take the reciprocal (1/x) of each datum). Chris ... Christopher D Green Department of Psychology York University Toronto, ON M6C 1G4 chri...@yorku.ca http://www.yorku.ca/christo On Nov 11, 2013, at 1:01 PM, Michael Britt mich...@thepsychfiles.com wrote: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person (or there's some form of cognitive dissonance going on, but that's not my question. No matter how big the sample size (mine was 160 respondents) I assume you'll always get a skewed distribution on a question like this so wouldn't I be breaking the normalization assumption if I were to do correlations using these results? I assume I could either do: a) do some kind of transformation - but I've never done one before so I’m not familiar with it, or b) recode the data into 3 categories (perhaps 1-5 is low satisfaction, 6-7 is moderate and 8-10 is high) and do a chi-squre instead of a correlation. Any thoughts? Appreciate it. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: chri...@yorku.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=430248.781165b5ef80a3cd2b14721caf62b d92n=Tl=tipso=30023 or send a blank email to leave-30023-430248.781165b5ef80a3cd2b14721caf62bd92@fsulist.frostburg. edu --- You are currently subscribed to tips as: j.cl...@uwinnipeg.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13251.645f86b5cec4da0a56ffea7a891720c9n=Tl=tipso=30040 or send a blank email to leave-30040-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30042 or send a blank email to leave-30042-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
Re: [tips] What to do with skewed data
Thanks for all these suggestions. I've been thinking of trying all or most of them. I assume that I would get slightly different results, so let me ask this: what criteria would you use to determine which transformation gave the best result? Would it be the one with the least amount of skew/most normal in distribution? Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt On Nov 11, 2013, at 7:04 PM, Jim Clark j.cl...@uwinnipeg.ca wrote: Hi People have been noting various transformations, depending on reversing or not the original. In essence many common transformations can be conceptualized as the original scores raised to various powers, with the powers being greater than or less than 1. x^-1 = reciprocal 1/x x^~0 = logarithmic x^.5 = square root x^1 = original x^2 = ... I'm mostly used to thinking of these in terms of various non-linear relationships (powers 1 compress upper end, powers 1 expand upper end), but some of these will increase skewness and others will decrease it, depending on the direction of skew. Possible to experiment with them to observe effect. Perhaps also worth plotting some of the relationships you are interested in? Take care Jim Jim Clark Professor Chair of Psychology 204-786-9757 4L41A -Original Message- From: Christopher Green [mailto:chri...@yorku.ca] Sent: Monday, November 11, 2013 5:31 PM To: Teaching in the Psychological Sciences (TIPS) Subject: Re: [tips] What to do with skewed data Michael, There are couple of standard ways to transform skewed data. Invert the data (subtract each datum from one greater than the highest value) so that the skew is positive. Then, Depending of the strength of the skew, do a square root or logarithmic transformation. Alternatively, don't invert it and take the reciprocal (1/x) of each datum). Chris ... Christopher D Green Department of Psychology York University Toronto, ON M6C 1G4 chri...@yorku.ca http://www.yorku.ca/christo On Nov 11, 2013, at 1:01 PM, Michael Britt mich...@thepsychfiles.com wrote: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person (or there's some form of cognitive dissonance going on, but that's not my question. No matter how big the sample size (mine was 160 respondents) I assume you'll always get a skewed distribution on a question like this so wouldn't I be breaking the normalization assumption if I were to do correlations using these results? I assume I could either do: a) do some kind of transformation - but I've never done one before so I’m not familiar with it, or b) recode the data into 3 categories (perhaps 1-5 is low satisfaction, 6-7 is moderate and 8-10 is high) and do a chi-squre instead of a correlation. Any thoughts? Appreciate it. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: chri...@yorku.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=430248.781165b5ef80a3cd2b14721caf62b d92n=Tl=tipso=30023 or send a blank email to leave-30023-430248.781165b5ef80a3cd2b14721caf62bd92@fsulist.frostburg. edu --- You are currently subscribed to tips as: j.cl...@uwinnipeg.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13251.645f86b5cec4da0a56ffea7a891720c9n=Tl=tipso=30040 or send a blank email to leave-30040-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu --- You are currently subscribed to tips as: michael.br...@thepsychfiles.com. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13405.0125141592fa9ededc665c55d9958f69n=Tl=tipso=30042 or send a blank email to leave-30042-13405.0125141592fa9ededc665c55d9958...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30043 or send a blank email to leave-30043-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
Re: [tips] What to do with skewed data
You might use something like a Q-Q plot to see which gives you the closest to a normal distribution. David Howell's big textbook shows you how. Chris ... Christopher D Green Department of Psychology York University Toronto, ON M6C 1G4 chri...@yorku.ca http://www.yorku.ca/christo On Nov 11, 2013, at 7:26 PM, Michael Britt mich...@thepsychfiles.com wrote: Thanks for all these suggestions. I've been thinking of trying all or most of them. I assume that I would get slightly different results, so let me ask this: what criteria would you use to determine which transformation gave the best result? Would it be the one with the least amount of skew/most normal in distribution? Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt On Nov 11, 2013, at 7:04 PM, Jim Clark j.cl...@uwinnipeg.ca wrote: Hi People have been noting various transformations, depending on reversing or not the original. In essence many common transformations can be conceptualized as the original scores raised to various powers, with the powers being greater than or less than 1. x^-1 = reciprocal 1/x x^~0 = logarithmic x^.5 = square root x^1 = original x^2 = ... I'm mostly used to thinking of these in terms of various non-linear relationships (powers 1 compress upper end, powers 1 expand upper end), but some of these will increase skewness and others will decrease it, depending on the direction of skew. Possible to experiment with them to observe effect. Perhaps also worth plotting some of the relationships you are interested in? Take care Jim Jim Clark Professor Chair of Psychology 204-786-9757 4L41A -Original Message- From: Christopher Green [mailto:chri...@yorku.ca] Sent: Monday, November 11, 2013 5:31 PM To: Teaching in the Psychological Sciences (TIPS) Subject: Re: [tips] What to do with skewed data Michael, There are couple of standard ways to transform skewed data. Invert the data (subtract each datum from one greater than the highest value) so that the skew is positive. Then, Depending of the strength of the skew, do a square root or logarithmic transformation. Alternatively, don't invert it and take the reciprocal (1/x) of each datum). Chris ... Christopher D Green Department of Psychology York University Toronto, ON M6C 1G4 chri...@yorku.ca http://www.yorku.ca/christo On Nov 11, 2013, at 1:01 PM, Michael Britt mich...@thepsychfiles.com wrote: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person (or there's some form of cognitive dissonance going on, but that's not my question. No matter how big the sample size (mine was 160 respondents) I assume you'll always get a skewed distribution on a question like this so wouldn't I be breaking the normalization assumption if I were to do correlations using these results? I assume I could either do: a) do some kind of transformation - but I've never done one before so I’m not familiar with it, or b) recode the data into 3 categories (perhaps 1-5 is low satisfaction, 6-7 is moderate and 8-10 is high) and do a chi-squre instead of a correlation. Any thoughts? Appreciate it. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: chri...@yorku.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=430248.781165b5ef80a3cd2b14721caf62b d92n=Tl=tipso=30023 or send a blank email to leave-30023-430248.781165b5ef80a3cd2b14721caf62bd92@fsulist.frostburg. edu --- You are currently subscribed to tips as: j.cl...@uwinnipeg.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13251.645f86b5cec4da0a56ffea7a891720c9n=Tl=tipso=30040 or send a blank email to leave-30040-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu --- You are currently subscribed to tips as: michael.br...@thepsychfiles.com. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13405.0125141592fa9ededc665c55d9958f69n=Tl=tipso=30042 or send a blank email to leave-30042-13405.0125141592fa9ededc665c55d9958...@fsulist.frostburg.edu --- You are currently subscribed to tips as: chri...@yorku.ca. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=430248.781165b5ef80a3cd2b14721caf62bd92n=Tl=tipso=30043 or send a blank email to leave-30043-430248.781165b5ef80a3cd2b14721caf62b...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here:
Re: [tips] What to do with skewed data
On Mon, 11 Nov 2013 10:31:19 -0800, David Epstein wrote: On Mon, 11 Nov 2013, Michael Britt went: I did a survey which asked respondents how satisfied they are in their current (romantic) relationship on a 1=10 point scale (where 10=very satisfied). While there was some variation, not surprisingly, the results are strongly negatively skewed. That makes sense - most people are probably satisfied with their relationships or they would leave the other person So you've got yourself the mirror image of one of these: http://psycnet.apa.org/psycinfo/1983-09462-001 |The insidious L-shaped distribution. |Bradley, James V. |Bulletin of the Psychonomic Society, Vol 20(2), Aug 1982, 85-88. | |L-shaped distributions are probably more prevalent than generally |realized. They are highly conducive to nonrobustness of |normality-assuming statistical tests, and strongly resist |transformation to normality. The thinner the tail of the |distribution, the more unlikely it is that its L-shapedness will be |detected by inspecting a sample drawn from it. Yet, as the tail of an |L-shaped distribution becomes increasingly shallow, its skewness and |kurtosis depart increasingly from their normal-distribution values, |and the distribution becomes increasingly conducive to drastic |nonrobustness. Worse, a fairly common type of experimental situation |in psychological research produces shallow-tailed L-shaped |distributions. I have to applaud David's ability to locate obscure journal articles. ;-) I remember this article from graduate school because a Michigan math psychologist referred to it in a class when discussing nonparametric analysis (I should note, however, that his teaching was so atrocious that he was limited to teaching graduate students -- I took a class or two with him and can honestly say I did not understand anything he said, indeed, at times *he* did not seem to know what he was saying but that's another story). Bradley wrote a book on nonparametric analysis titled Distribution Free Statistics which my former teacher was fond of which one might consult: http://books.google.com/books?id=NB-rnQEACAAJdq=%22James+V.+Bradley%22++nonparametrichl=ensa=Xei=B4-BUsdN7uewBL3UgsAPved=0CIMBEOgBMBE If you do a search on statistical techniques for zero-inflated continuous [or semicontinuous] data, you might be able to apply them to your ten-inflated data. As others have already mentioned in this thread, one could transform the data and I would suggest taking a look at Tukey's Ladder of Powers in his Exploratory Data Analysis (the EDA book). One summary of this is provided on this website: http://onlinestatbook.com/2/transformations/tukey.html However, I think a couple of questions need to be answered before a meaningful analysis is attempted. Consider: (1) Does one really believe that these numbers come from a normal population distribution? Transforming these makes this assumption. It also makes the assumption that the instrument used suffers from a ceiling effect, that is, just as with a multiple choice test that is too easy, the nature of the questions would not allow a normal distribution to appear in the sample -- assuming that they really do come from a normal distribution. If one has the time and inclination, one can view this as a case of the sample data having a truncated normal distribution, a situation that has been studied by statisticians and economists; see: http://econpapers.repec.org/article/ecmemetrp/v_3a41_3ay_3a1973_3ai_3a6_3ap_3a997-1016.htm This paper by Amemiya is rather old may have been superceded by newer techniques. (2) David's suggestion above seems to suggest that alternative distributions should be considered. There is the zero inflated Poisson distribution but, if memory serves, that is for discrete values. You would have to reverse code the scale and other stuff. But there may be other more appropriate distributions that one might consider. I guess this situation reinforces the need for one to know what population distribution one is collecting one's data from. ;-) -Mike Palij New York University m...@nyu.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30046 or send a blank email to leave-30046-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
RE: [tips] What to do with skewed data
Hi Go to the very bottom of the link that Michael P provided below and you will find a brief discussion of skewness and a VERY nice demo of the impact of different powers on a skewed distribution (pretty much the mirror image of Michael B's in the original posting). Michael P As others have already mentioned in this thread, one could transform the data and I would suggest taking a look at Tukey's Ladder of Powers in his Exploratory Data Analysis (the EDA book). One summary of this is provided on this website: http://onlinestatbook.com/2/transformations/tukey.html Take care Jim --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5n=Tl=tipso=30050 or send a blank email to leave-30050-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
