[R] help on quantile
Hi, I would like to use the quantile function, but I have some doubts to choose the value of type parameter. Does this depend on the shape/structure of my distribution? I cannot unde rstand what mean the expressions Discontinuous sample quantile and Continuous sample quantile, respectively. My distributions (histograms) sometimes have some holes, that is I have some bins with zero samples. Someone can help me, please? thank you! giov -- View this message in context: http://www.nabble.com/help-on-quantile-tp19921087p19921087.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] help on wavelet
Hi, I have little experience using wavelet and I would like to know if it is possible,using R wavelet package, to have a plot of frequency versus time. thank you giov -- View this message in context: http://www.nabble.com/help-on-wavelet-tp19395583p19395583.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] help on jarque test
Hi all, I used the function jarque.test (in the moments package) on my data set and I obtained something like this: Jarque-Bera Normality Test data: x JB = 4.8381, p-value = 0.089 alternative hypothesis: greater or Jarque-Bera Normality Test data: x JB = 2.6018, p-value = 0.2723 alternative hypothesis: greater I cannot understand this. Please, someone can help me? thank you giov -- View this message in context: http://www.nabble.com/help-on-jarque-test-tp19312270p19312270.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] R update
Hi all, please, someone can explain me how update my R version? thank you! giov -- View this message in context: http://www.nabble.com/R-update-tp19291451p19291451.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] testing symmetry of distibution
Hi all, I need to test symmetry of distributions. How can I do it using R? thank you! giov -- View this message in context: http://www.nabble.com/testing-symmetry-of-distibution-tp19028200p19028200.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] dixon test
Steve,
thank you so much for very very useful helps and comments!
for my research I used a surrogate approach (number of surrogates=100) and I
made a comparison among index values (very similar to a correlation index
computed with a second known signal ) from my original data and from
surrogates data , respectively. The idea is to find that the index value
from the original data is very different (in statistical sense)from those
surrogates. To evaluate this difference I thought that the best thing to do
it was the evaluation if the index from original data can be considered an
outlier value in comparison with the . Is it a correct approach?
thank you!!
giov
S Ellison wrote:
giov,
It sounds like you have approximately symmetric distributions. If that
is so, and particularly if the standard deviation is less than about 20%
of the mean, I'll stick my neck out and say I would assume underlying
normality for outlier testing purposes unless there's a reason to do
otherwise (eg if you're testing variances, normality would _not_ be a
good assumption!).
The reason I'd do that is that is that it should not make a big
difference to the outcome with near-symmetric distributions. If it does,
your 'outliers' are borderline anyway.
Similarly, although folk can get quite exercised over which test to use
and what significance level to choose, the test you use isn't very
important either, as long as the intention is just to screen data to
make sure the most influential/extreme points are not mistakes.
Given that, you can use any of the tests in library(outliers). You can
also use boxplot.stats, and look at the $out list, like
y-c(rnorm(15,10), 25.1) #25.1 should be an outlier
(bxs-boxplot.stats(y))
#and locate the outliers in y:
which(y %in% bxs$out)
Another useful approach is to use robust estimates of mean and
dispersion, like hubers() in the MASS package, and then calculate simple
scores, with a z-like cutoff to identify outliers:
require(MASS)
hy-hubers(y)
hscore-(y-hy$mu)/hy$s
which(abs(hscore)3)
Using the 'mad' or iqr options in outliers::scores will be broadly
similar in outcome.
Most of the modelling tools in R also offer useful diagnostics for
'odd' points. I find examining the residuals from rlm in MASS
particularly useful if you're seeking outliers in a regression context.
A more important question is what you will do if you find any outliers.
Outliers are just unusual compared to some expectation, not
automatically 'wrong'. Screening data for anomalies is good practice;
checking them to make sure they aren't mistakes is to be encouraged;
correcting mistakes if you find them is a no-brainer. But throwing
outliers away is something to think about very carefully, and on a
case-by-case basis. Sometimes, outliers are a genuine feature of the
process under study, or even the 'interesting' parts of the data. It's
generally unsafe to throw them out without good reason.
Steve E
PS: Contrary to my earlier confident assertion of the non-existence of
nonparametric outlier tests, Barnett and Lewis DOES include some general
suggestions on 'nonparametric' outlier testing. But it also includes the
note that this ... smacks of throwing out the bathwater before the baby
has even been immersed. I guess they don't think much of the idea
either.
giov [EMAIL PROTECTED] 13/08/2008 15:21:25
Thank you so much, I have not much experience on outliers =), I thought
that
there were nonparametric distribution-free outliers test =(. What is
the
most general distribution I can use? I did histogram of my data set
and
sometimes normal distribution seems to occur, sometimes an uniform
distribution seems to occur. So, I cannot understand what distribution
I can
use for my whole data set
***
This email and any attachments are confidential. Any use...{{dropped:8}}
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
View this message in context:
http://www.nabble.com/dixon-test-tp18940260p18980152.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] dixon test
Hi, thank you very much for your useful help =). just a question...I don't know what is the distribution of my data (normal, T, etc...). So, how can I set the type parameter? There is a type value to use in case of a distribution-free statistical test? Thank you so much! Fernando Marmolejo-Ramos wrote: hi giov about the dixon test... i just run a simple test with a sample of 40 and I got: Error in dixon.test(x) : Sample size must be in range 3-30 So it seems that most of the test in the outliers package are designed for small samples. See also the Rnews article published in May 2006 (vol 6/2) called processing data for outliers by Lukasz Komsta (the developer of the package). However there is in that package a function called scores which works for big samples. You can also see the p-values and z scores for the observations you have and determine which values are considered outliers. Try this simple syntax: library(outliers) library(gamlss.dist) # this produces a exponential+Gaussian distribution (which usually has heaps of outliers!) x - rexGAUS(100,2000,3000,5000) # this confirms that Dixon works for samples between 3 and 30!!! dixon.test(x) # just to see what the data set looks like and visually confirm the outliers boxplot(x, notch=T) # sort the scores in ascending order sort(x) # returns probability of each score (using z scores) to be an outlier in order sort(scores(x, type=z, prob=1)) # determines which scores are considered outliers with a 95% confidence sort(scores(x, prob=0.95)) The author points regarding the prob part... prob If set, the corresponding p-values instead of scores are given. If value is set to 1, p-value are returned. Otherwise, a logical vector is formed, indicating which values are exceeding specified probability. In z and mad types, there is also possibility to set this value to zero, and then scores are confirmed to (n-1)/sqrt(n) value, according to Shiffler (1998). The iqr type does not support probabilities, but lim value can be specified. The reference of Shiffler is not as the one that appears in the help. It is this one: Schiffler, R.E (1988). Maximum Z scores and outliers. Am. Stat. 42, 1, 79-80. I hope this helps, Fernando -- View this message in context: http://www.nabble.com/dixon-test-tp18940260p18960162.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] dixon test
Thank you so much, I have not much experience on outliers =), I thought that
there were nonparametric distribution-free outliers test =(. What is the
most general distribution I can use? I did histogram of my data set and
sometimes normal distribution seems to occur, sometimes an uniform
distribution seems to occur. So, I cannot understand what distribution I can
use for my whole data set
S Ellison wrote:
giov [EMAIL PROTECTED] 13/08/2008 10:59:32
just a question...I don't know
what is the distribution of my data (normal, T, etc...). So, how can I
set
the type parameter?
You must assume an underlying distribution or you can't do an outlier
test.
Outliers are just unusually extreme data points. They can only be
considered 'unusual' if there is some basis - a distribution assumption
- for deciding what is 'usual'. The assumed underlying distribution
describes what is expected to be 'usual'.
With no distribution assumption, there is no basis for considering any
data point unusual, so the idea of an outlier really has no meaning.
Steve E
***
This email and any attachments are confidential. Any use...{{dropped:8}}
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
View this message in context:
http://www.nabble.com/dixon-test-tp18940260p18964049.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] dixon test
Hi, I need some help using the R outliers package. I would like to perform a Q-test (Dixon test) on my data set. I used the dixon.test function, but I cannot understand what is the confidence level used to perform the test. I have n=101 (n= number of data). So, can I use directly dixon.test ? What about qdixon and qtable functions? thank you so much! -- View this message in context: http://www.nabble.com/dixon-test-tp18940260p18940260.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
