Re: [Rd] [R] Bug : Autocorrelation in sample drawn from stats::rnorm (hmh)
On 05/10/2018, 09:45, "R-help on behalf of hmh" wrote: Hi, Thanks William for this fast answer, and sorry for sending the 1st mail to r-help instead to r-devel. I noticed that bug while I was simulating many small random walks using c(0,cumsum(rnorm(10))). Then the negative auto-correlation was inducing a muchsmaller space visited by the random walks than expected if there would be no auto-correlation in the samples. The code I provided and you optimized was only provided to illustrated and investigate that bug. It is really worrying that most of the R distributions are affected by this bug What I did should have been one of the first check done for _*each*_ distributions by the developers of these functions ! And if as you suggested this is a "tolerated" _error_ of the algorithm, I do think this is a bad choice, but any way, this should have been mentioned in the documentations of the functions !! cheers, hugo This is not a bug. You have simply rediscovered the finite-sample bias in the sample autocorrelation coefficient, known at least since Kendall, M. G. (1954). Note on bias in the estimation of autocorrelation. Biometrika, 41(3-4), 403-404. The bias is approximately -1/T, with T sample size, which explains why it seems to disappear in the larger sample sizes you consider. Jan __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Bug : Autocorrelation in sample drawn from stats::rnorm (hmh)
I got it ! thanks and sorry for annoying you with that. have a nice day, hugo On 05/10/2018 11:16, Deepayan Sarkar wrote: > On Fri, Oct 5, 2018 at 2:07 PM hmh wrote: >> On 05/10/2018 10:28, Annaert Jan wrote: >>> you discard any time series structure; >> But that is PRECISELY what a call a bug: >> There should not be any "time series structure" in the output or rnorm, >> runif and so on but there is one. >> >> rnorm(N,0,1) >> should give on average the same output as >> sample(rnorm(N,0,1)) > Agreed, but that is not what your code is testing. You seem to think > that something much more specific should be true; namely, > > X[1:10] ~ iid normal, then > > cor(X[1:9], X[2:10]) > > and > > cor(sample(X[-1]), sample(X[-10])) > > should have the same distribution. This is not at all obvious, and in > fact not true. > > Please check the reference you have been pointed to. Here is a related > article in the same volume: > > https://www.jstor.org/stable/2332719 > > -Deepayan > > >> Which is not the case. rnorm(N,0,1) should draw INDEPENDENT samples i.e. >> without time series structure ! >> >> >> -- >> - no title specified >> >> Hugo Mathé-Hubert >> >> ATER >> >> Laboratoire Interdisciplinaire des Environnements Continentaux (LIEC) >> >> UMR 7360 CNRS - Bât IBISE >> >> Université de Lorraine - UFR SciFA >> >> 8, Rue du Général Delestraint >> >> F-57070 METZ >> >> +33(0)9 77 21 66 66 >> - - - - - - - - - - - - - - - - - - >> Les réflexions naissent dans les doutes et meurent dans les certitudes. >> Les doutes sont donc un signe de force et les certitudes un signe de >> faiblesse. La plupart des gens sont pourtant certains du contraire. >> - - - - - - - - - - - - - - - - - - >> Thoughts appear from doubts and die in convictions. Therefore, doubts >> are an indication of strength and convictions an indication of weakness. >> Yet, most people believe the opposite. >> >> >> [[alternative HTML version deleted]] >> >> __ >> r-h...@r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. -- - no title specified Hugo Mathé-Hubert ATER Laboratoire Interdisciplinaire des Environnements Continentaux (LIEC) UMR 7360 CNRS - Bât IBISE Université de Lorraine - UFR SciFA 8, Rue du Général Delestraint F-57070 METZ +33(0)9 77 21 66 66 - - - - - - - - - - - - - - - - - - Les réflexions naissent dans les doutes et meurent dans les certitudes. Les doutes sont donc un signe de force et les certitudes un signe de faiblesse. La plupart des gens sont pourtant certains du contraire. - - - - - - - - - - - - - - - - - - Thoughts appear from doubts and die in convictions. Therefore, doubts are an indication of strength and convictions an indication of weakness. Yet, most people believe the opposite. [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Bug : Autocorrelation in sample drawn from stats::rnorm (hmh)
On Fri, Oct 5, 2018 at 2:07 PM hmh wrote: > > On 05/10/2018 10:28, Annaert Jan wrote: > > you discard any time series structure; > But that is PRECISELY what a call a bug: > There should not be any "time series structure" in the output or rnorm, > runif and so on but there is one. > > rnorm(N,0,1) > should give on average the same output as > sample(rnorm(N,0,1)) Agreed, but that is not what your code is testing. You seem to think that something much more specific should be true; namely, X[1:10] ~ iid normal, then cor(X[1:9], X[2:10]) and cor(sample(X[-1]), sample(X[-10])) should have the same distribution. This is not at all obvious, and in fact not true. Please check the reference you have been pointed to. Here is a related article in the same volume: https://www.jstor.org/stable/2332719 -Deepayan > Which is not the case. rnorm(N,0,1) should draw INDEPENDENT samples i.e. > without time series structure ! > > > -- > - no title specified > > Hugo Mathé-Hubert > > ATER > > Laboratoire Interdisciplinaire des Environnements Continentaux (LIEC) > > UMR 7360 CNRS - Bât IBISE > > Université de Lorraine - UFR SciFA > > 8, Rue du Général Delestraint > > F-57070 METZ > > +33(0)9 77 21 66 66 > - - - - - - - - - - - - - - - - - - > Les réflexions naissent dans les doutes et meurent dans les certitudes. > Les doutes sont donc un signe de force et les certitudes un signe de > faiblesse. La plupart des gens sont pourtant certains du contraire. > - - - - - - - - - - - - - - - - - - > Thoughts appear from doubts and die in convictions. Therefore, doubts > are an indication of strength and convictions an indication of weakness. > Yet, most people believe the opposite. > > > [[alternative HTML version deleted]] > > __ > r-h...@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
