Dear Kind List, I am still battling with this. I have, however, made some progress with the suggestions of Micheal and others. At least, I have a better picture of what I want to do now as I will attempt a detailed description here.
I am aware I should show you just a small part of my code and data. But when I copied out a small portion and run to see what you get when I send that, I was not satisfied with the signal displayed. The epoch analysis averages data and is quite sensitive to leveraging, especially if a small sample is used. So please permit/exercise patience me to display the series of epoch that give the averaged valued used. You can just run the code and see the signal of interest. Here is the code and the data: dta <- read.table( text ="n CR -5 8969 -4 8932 -3 8929 -2 8916 -1 8807 0 8449 1 8484 2 8148 3 8282 4 8305 5 8380 6 8530 7 8642 8 8780 9 8890 10 8962 -5 8929 -4 8916 -3 8807 -2 8449 -1 8484 0 8148 1 8282 2 8305 3 8380 4 8530 5 8642 6 8780 7 8890 8 8962 9 8949 10 8974 -5 8744 -4 8786 -3 8828 -2 8807 -1 8716 0 8520 1 8634 2 8640 3 8636 4 8658 5 8699 6 8682 7 8621 8 8626 9 8660 10 8737 -5 8592 -4 8612 -3 8628 -2 8589 -1 8318 0 8264 1 8294 2 8410 3 8442 4 8416 5 8389 6 8412 7 8453 8 8563 9 8581 10 8613 -5 8264 -4 8294 -3 8410 -2 8442 -1 8416 0 8389 1 8412 2 8453 3 8563 4 8581 5 8613 6 8647 7 8613 8 8508 9 7829 10 7499 -5 8613 -4 8647 -3 8613 -2 8508 -1 7829 0 7499 1 8213 2 7993 3 7821 4 8316 5 8460 6 8533 7 8584 8 8586 9 8567 10 8573 -5 8508 -4 7829 -3 7499 -2 8213 -1 7993 0 7821 1 8316 2 8460 3 8533 4 8584 5 8586 6 8567 7 8573 8 8617 9 8591 10 8661 -5 8851 -4 8893 -3 8858 -2 8803 -1 8790 0 8468 1 8545 2 8570 3 8568 4 8624 5 8669 6 8236 7 8190 8 8313 9 8389 10 8421 -5 8803 -4 8790 -3 8468 -2 8545 -1 8570 0 8568 1 8624 2 8669 3 8236 4 8190 5 8313 6 8389 7 8421 8 8468 9 8537 10 8580 -5 8570 -4 8568 -3 8624 -2 8669 -1 8236 0 8190 1 8313 2 8389 3 8421 4 8468 5 8537 6 8580 7 8605 8 8646 9 8690 10 8770 -5 8690 -4 8770 -3 8799 -2 8821 -1 8666 0 8539 1 8633 2 8617 3 8651 4 8693 5 8715 6 8738 7 8716 8 8677 9 8680 10 8700 -5 8756 -4 8632 -3 8662 -2 8596 -1 8552 0 8502 1 8633 2 8702 3 8745 4 8730 5 8708 6 8817 7 8724 8 8688 9 8693 10 8746 -5 8926 -4 8888 -3 8798 -2 8651 -1 8678 0 8578 1 8593 2 8598 3 8526 4 8181 5 8204 6 8373 7 8599 8 8773 9 8784 10 8746 -5 8678 -4 8578 -3 8593 -2 8598 -1 8526 0 8181 1 8204 2 8373 3 8599 4 8773 5 8784 6 8746 7 8747 8 8757 9 8749 10 8767 -5 8757 -4 8749 -3 8767 -2 8754 -1 8695 0 8631 1 8661 2 8653 3 8588 4 8562 5 8613 6 8595 7 8498 8 8404 9 8507 10 8599 -5 8695 -4 8631 -3 8661 -2 8653 -1 8588 0 8562 1 8613 2 8595 3 8498 4 8404 5 8507 6 8599 7 8592 8 8600 9 8637 10 8635 -5 8588 -4 8562 -3 8613 -2 8595 -1 8498 0 8404 1 8507 2 8599 3 8592 4 8600 5 8637 6 8635 7 8632 8 8674 9 8644 10 8687 -5 8595 -4 8498 -3 8404 -2 8507 -1 8599 0 8592 1 8600 2 8637 3 8635 4 8632 5 8674 6 8644 7 8687 8 8721 9 8747 10 8748 -5 8599 -4 8592 -3 8600 -2 8637 -1 8635 0 8632 1 8674 2 8644 3 8687 4 8721 5 8747 6 8748 7 8739 8 8763 9 8792 10 8558 -5 8600 -4 8637 -3 8635 -2 8632 -1 8674 0 8644 1 8687 2 8721 3 8747 4 8748 5 8739 6 8763 7 8792 8 8558 9 8442 10 8555 -5 8748 -4 8739 -3 8763 -2 8792 -1 8558 0 8442 1 8555 2 8622 3 8634 4 8698 5 8732 6 8713 7 8732 8 8681 9 8615 10 8624 -5 8698 -4 8732 -3 8713 -2 8732 -1 8681 0 8615 1 8624 2 8649 3 8656 4 8678 5 8723 6 8693 7 8548 8 7803 9 7801 10 7724 -5 8723 -4 8693 -3 8548 -2 7803 -1 7801 0 7724 1 7910 2 7829 3 7995 4 8156 5 8307 6 8377 7 8465 8 8506 9 8516 10 8536 -5 8548 -4 7803 -3 7801 -2 7724 -1 7910 0 7829 1 7995 2 8156 3 8307 4 8377 5 8465 6 8506 7 8516 8 8536 9 8574 10 8623 -5 8821 -4 8856 -3 8798 -2 8772 -1 8705 0 8682 1 8691 2 8720 3 8727 4 8789 5 8821 6 8811 7 8841 8 8849 9 8849 10 8860 -5 8835 -4 8829 -3 8826 -2 8799 -1 8775 0 8756 1 8793 2 8814 3 8847 4 8838 5 8833 6 8841 7 8847 8 8903 9 8933 10 8918 -5 8890 -4 8875 -3 8874 -2 8865 -1 8891 0 8839 1 8853 2 8888 3 8884 4 8890 5 8889 6 8839 7 8879 8 8908 9 8924 10 8882 -5 8853 -4 8888 -3 8884 -2 8890 -1 8889 0 8839 1 8879 2 8908 3 8924 4 8882 5 8910 6 8903 7 8859 8 8858 9 8863 10 8847 -5 8924 -4 8882 -3 8910 -2 8903 -1 8859 0 8858 1 8863 2 8847 3 8883 4 8869 5 8878 6 8897 7 8922 8 8895 9 8858 10 8858 -5 8910 -4 8903 -3 8859 -2 8858 -1 8863 0 8847 1 8883 2 8869 3 8878 4 8897 5 8922 6 8895 7 8858 8 8858 9 8736 10 8905 -5 8859 -4 8858 -3 8863 -2 8847 -1 8883 0 8869 1 8878 2 8897 3 8922 4 8895 5 8858 6 8858 7 8736 8 8905 9 8935 10 8974 -5 8897 -4 8922 -3 8895 -2 8858 -1 8858 0 8736 1 8905 2 8935 3 8974 4 8946 5 8952 6 9010 7 8980 8 8976 9 8970 10 8961 -5 9376 -4 9336 -3 9311 -2 9287 -1 9221 0 9087 1 9132 2 9175 3 9166 4 9240 5 9264 6 9271 7 9319 8 9324 9 9333 10 9351 -5 9287 -4 9221 -3 9087 -2 9132 -1 9175 0 9166 1 9240 2 9264 3 9271 4 9319 5 9324 6 9333 7 9351 8 9362 9 9385 10 9354 -5 9407 -4 9414 -3 9354 -2 9298 -1 9319 0 9147 1 9178 2 9196 3 9258 4 9303 5 9369 6 9382 7 9375 8 9389 9 9376 10 9264 -5 9386 -4 9396 -3 9424 -2 9391 -1 9284 0 9267 1 9278 2 9318 3 9334 4 9275 5 9306 6 9308 7 9358 8 9335 9 9373 10 9379 -5 9284 -4 9267 -3 9278 -2 9318 -1 9334 0 9275 1 9306 2 9308 3 9358 4 9335 5 9373 6 9379 7 9355 8 9340 9 9327 10 9320 -5 9327 -4 9320 -3 9315 -2 9336 -1 9371 0 9259 1 9330 2 9355 3 9334 4 9353 5 9370 6 9394 7 9400 8 9318 9 9037 10 8994 -5 9394 -4 9400 -3 9318 -2 9037 -1 8994 0 8943 1 8964 2 8997 3 9158 4 8964 5 8564 6 8736 7 8818 8 8938 9 9034 10 9132 -5 8943 -4 8964 -3 8997 -2 9158 -1 8964 0 8564 1 8736 2 8818 3 8938 4 9034 5 9132 6 9167 7 9200 8 9257 9 9266 10 9306 -5 9338 -4 9354 -3 9372 -2 9338 -1 9308 0 9282 1 9324 2 9318 3 9342 4 9370 5 9331 6 9327 7 9338 8 9381 9 9394 10 9332 -5 9372 -4 9338 -3 9308 -2 9282 -1 9324 0 9318 1 9342 2 9370 3 9331 4 9327 5 9338 6 9381 7 9394 8 9332 9 9331 10 9293 -5 9338 -4 9381 -3 9394 -2 9332 -1 9331 0 9293 1 9309 2 9325 3 9406 4 9409 5 9413 6 9426 7 9440 8 9449 9 9512 10 9494 -5 9361 -4 9354 -3 9299 -2 9282 -1 9250 0 9242 1 9254 2 9321 3 9390 4 9414 5 9435 6 9437 7 9426 8 9398 9 9383 10 9354 -5 9365 -4 9421 -3 9416 -2 9355 -1 9338 0 9324 1 9325 2 9322 3 9319 4 9381 5 9315 6 9314 7 9359 8 9403 9 9419 10 9474 -5 9355 -4 9338 -3 9324 -2 9325 -1 9322 0 9319 1 9381 2 9315 3 9314 4 9359 5 9403 6 9419 7 9474 8 9525 9 9501 10 9447 -5 9325 -4 9322 -3 9319 -2 9381 -1 9315 0 9314 1 9359 2 9403 3 9419 4 9474 5 9525 6 9501 7 9447 8 9424 9 9396 10 9388 -5 9447 -4 9424 -3 9396 -2 9388 -1 9396 0 9346 1 9358 2 9353 3 9350 4 9378 5 9372 6 9354 7 9349 8 9392 9 9440 10 9467 -5 9388 -4 9396 -3 9346 -2 9358 -1 9353 0 9350 1 9378 2 9372 3 9354 4 9349 5 9392 6 9440 7 9467 8 9519 9 9550 10 9565 -5 9353 -4 9350 -3 9378 -2 9372 -1 9354 0 9349 1 9392 2 9440 3 9467 4 9519 5 9550 6 9565 7 9565 8 9497 9 9500 10 9472 -5 9522 -4 9529 -3 9492 -2 9432 -1 9382 0 9355 1 9361 2 9350 3 9382 4 9451 5 9491 6 9506 7 9529 8 9543 9 9556 10 9553 -5 9492 -4 9432 -3 9382 -2 9355 -1 9361 0 9350 1 9382 2 9451 3 9491 4 9506 5 9529 6 9543 7 9556 8 9553 9 9502 10 9470 -5 9551 -4 9505 -3 9389 -2 9406 -1 9377 0 9284 1 9365 2 9424 3 9412 4 9403 5 9384 6 9394 7 9404 8 9413 9 9407 10 9405 -5 9579 -4 9576 -3 9543 -2 9451 -1 9421 0 9361 1 9394 2 9400 3 9387 4 9366 5 9346 6 9360 7 9385 8 9435 9 9443 10 9430 -5 9361 -4 9394 -3 9400 -2 9387 -1 9366 0 9346 1 9360 2 9385 3 9435 4 9443 5 9430 6 9454 7 9531 8 9547 9 9581 10 9540 -5 9510 -4 9546 -3 9564 -2 9508 -1 9422 0 9369 1 9395 2 9438 3 9423 4 9392 5 9368 6 9366 7 9348 8 9340 9 9375 10 9391 -5 9423 -4 9392 -3 9368 -2 9366 -1 9348 0 9340 1 9375 2 9391 3 9466 4 9545 5 9574 6 9564 7 9527 8 9513 9 9494 10 9542 -5 9511 -4 9491 -3 9457 -2 9453 -1 9402 0 9382 1 9407 2 9437 3 9403 4 9404 5 9425 6 9486 7 9457 8 9451 9 9423 10 9401 -5 9425 -4 9486 -3 9457 -2 9451 -1 9423 0 9401 1 9429 2 9422 3 9431 4 9462 5 9475 6 9474 7 9487 8 9493 9 9495 10 9499 -5 9404 -4 9385 -3 9363 -2 9399 -1 9411 0 9355 1 9357 2 9363 3 9382 4 9387 5 9408 6 9429 7 9456 8 9487 9 9526 10 9487 -5 9493 -4 9439 -3 9400 -2 9378 -1 9371 0 9369 1 9374 2 9305 3 9298 4 9298 5 9325 6 9381 7 9477 8 9508 9 9496 10 9517 -5 9371 -4 9369 -3 9374 -2 9305 -1 9298 0 9298 1 9325 2 9381 3 9477 4 9508 5 9496 6 9517 7 9561 8 9570 9 9546 10 9544 -5 9510 -4 9506 -3 9530 -2 9441 -1 9427 0 9393 1 9420 2 9444 3 9468 4 9484 5 9525 6 9542 7 9557 8 9548 9 9550 10 9593 -5 9589 -4 9598 -3 9527 -2 9417 -1 9390 0 9374 1 9386 2 9407 3 9453 4 9447 5 9419 6 9386 7 9373 8 9364 9 9376 10 9389 -5 9453 -4 9447 -3 9419 -2 9386 -1 9373 0 9364 1 9376 2 9389 3 9376 4 9375 5 9370 6 9391 7 9458 8 9446 9 9456 10 9463 -5 9364 -4 9376 -3 9389 -2 9376 -1 9375 0 9370 1 9391 2 9458 3 9446 4 9456 5 9463 6 9500 7 9486 8 9474 9 9495 10 9531 -5 9491 -4 9441 -3 9388 -2 9380 -1 9369 0 9354 1 9367 2 9369 3 9341 4 9305 5 9308 6 9324 7 9385 8 9451 9 9496 10 9527 -5 9369 -4 9354 -3 9367 -2 9369 -1 9341 0 9305 1 9308 2 9324 3 9385 4 9451 5 9496 6 9527 7 9544 8 9543 9 9535 10 9536 -5 9586 -4 9583 -3 9572 -2 9533 -1 9454 0 9392 1 9420 2 9451 3 9475 4 9514 5 9561 6 9542 7 9502 8 9461 9 9468 10 9463 -5 9587 -4 9562 -3 9530 -2 9445 -1 9404 0 9395 1 9417 2 9449 3 9467 4 9470 5 9524 6 9512 7 9448 8 9398 9 9431 10 9467 -5 9467 -4 9470 -3 9524 -2 9512 -1 9448 0 9398 1 9431 2 9467 3 9490 4 9517 5 9526 6 9574 7 9573 8 9562 9 9563 10 9566 ",header=TRUE) data<-matrix(c(dta$CR),ncol=71) A<-matrix(rep(-5:10,71)) B<-matrix(data) oodf<-data.frame(A,B) a<--5:10 oodf<-data.frame(A,B) library(plotrix) std.error<-function(x) return(sd(x)/(sum(!is.na(x)))) oomean<-as.vector(by(oodf$B,oodf$A,mean)) oose<-as.vector(by(oodf$B,oodf$A,std.error)) plot(-5:10,oomean,type="l",ylim=c(8890,9100), ) A<-oomean-1.96*oose B<-oomean+1.96*oose lines(a,A,col="red") lines(a,B,col="red") My Question: I wish to conduct a randomization test of significance (90 and 99 percentile) of the reductions/decreases as displayed by the signal. I am attempting using: x<-sample(8890:9500,1136,replace=T ) to generate the random numbers, where 8890, 9500 and 1136 are the minimum and maximum of the signal and 1136 the length of sample data. Q1: Please how do I generate many samples as x above, say up to 5000 or 10,000? I manually generated and stored as x1,x2, x3 up to x100. Q2: Please how do I use this randomly generated numbers to test the statistical significance level of the signal generated by plot(-5:10,oomean,type="l",ylim=c(8890,9100), )? I wish to test for 90% and 99% percentile. I am sorry that this is too long. Many thanks for your kind contributions Best Ogbos On Sun, Feb 10, 2019 at 3:55 PM Ogbos Okike <giftedlife2...@gmail.com> wrote: > > Dear Michael, > This is great! Thank you. > > I have not really got any response other than yours. > > I have long before now included what I have in a paper submitted to a journal. > > I am awaiting the feedback of the reviewer. I will compare the > comments with your input here and determine the corrections to make > and probably return to the list for additional help. > > Best wishes > Ogbos > > On Fri, Feb 8, 2019 at 4:31 PM Meyners, Michael <meyner...@pg.com> wrote: > > > > Ogbos, > > > > You do not seem to have received a reply over the list yet, which might be > > due to the fact that this seems rather a stats than an R question. Neither > > got your attachment (Figure) through - see posting guide. > > > > I'm not familiar with epoch analysis, so not sure what exactly you are > > doing / trying to achieve, but some general thoughts: > > > > * You do NOT want to restrict your re-randomizations in a way that "none of > > the dates corresponds with the ones in the real event" - actually, as a > > general principle, the true data must be an admissible re-randomization as > > well. You seem to have excluded that (and a lot of other randomizations at > > the same time which might have occurred, i.e. dates 1 and 2 reversed but > > all others the same), thereby rendering the test invalid. Any restrictions > > you have on your re-randomizations must've applied to the original > > randomization as well. > > * If you have rather observational data (which I suspect, but not sure), > > Edgington & Onghena (2007) would rather refer to this as a permutation test > > - the difference being that you have to make strong assumptions (similar to > > parametric tests) on the nature of the data, which are designed-in to be > > true for randomization tests. It might be a merely linguistic > > discrimination, but it is important to note which assumptions have to be > > (implicitly) made. > > * I'm not sure what you mean by "mean differences" of the events - is that > > two groups you are comparing? If so, that seems reasonable, but just make > > sure the test statistic you use is reasonable and sensitive against the > > alternatives you are mostly interested in. The randomization/permutation > > test will never proof that, e.g., means are significantly different, but > > only that there is SOME difference. By selecting the appropriate test > > statistic, you can influence what will pop up more easily and what not, but > > you can never be sure (unless you make strong assumptions about everything > > else, like in many parametric tests). > > * For any test statistic, you would then determine the proportion of its > > values among the 5000 samples where it is as large or larger than the one > > observed (or as small or smaller, or either, depending on the nature of the > > test statistic and whether you aim for a one- or a two-sided test). That is > > your p value. If small enough, conclude significance. At least conceptually > > important: The observed test statistic is always part of the > > re-randomization (i.e. your 5000) - so you truly only do 4999 plus the one > > you observed. Otherwise the test may be more or less liberal. Your p value > > is hence no smaller than 1/n, where n is the total number of samples you > > looked at (including the observed one), a p value of 0 is not possible in > > randomization tests (nor in other tests, of course). > > > > I hope this is helpful, but you will need to go through these and refer to > > your own setup to check whether you adhered to the principles or not, which > > is impossible for me to judge based on the information provided (and I > > won't be able to look at excessive code to check either). > > > > Michael > > > > > -----Original Message----- > > > From: R-help <r-help-boun...@r-project.org> On Behalf Of Ogbos Okike > > > Sent: Montag, 28. Januar 2019 19:42 > > > To: r-help <r-help@r-project.org> > > > Subject: [R] Randomization Test > > > > > > Dear Contributors, > > > > > > I conducting epoch analysis. I tried to test the significance of my > > > result using > > > randomization test. > > > > > > Since I have 71 events, I randomly selected another 71 events, making sure > > > that none of the dates in the random events corresponds with the ones in > > > the real event. > > > > > > Following the code I found here > > > (https://www.uvm.edu/~dhowell/StatPages/R/RandomizationTestsWithR/R > > > andom2Sample/TwoIndependentSamplesR.html), > > > I combined these two data set and used them to generate another 5000 > > > events. I then plotted the graph of the mean differences for the 5000 > > > randomly generated events. On the graph, I indicated the region of the > > > mean difference between the real 71 epoch and the randomly selected 71 > > > epoch. > > > > > > Since the two tail test shows that the mean difference falls at the > > > extreme of > > > the randomly selected events, I concluded that my result is statistically > > > significant. > > > > > > > > > > > > I am attaching the graph to assistance you in you suggestions. > > > > > > I can attach both my code and the real and randomly generated events if > > > you > > > ask for it. > > > > > > My request is that you help me to understand if I am on the right track > > > or no. > > > This is the first time I am doing this and except the experts decide, I > > > am not > > > quite sure whether I am right or not. > > > > > > Many thanks for your kind concern. > > > > > > Best > > > Ogbos > > > ______________________________________________ > > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > PLEASE do read the posting guide http://www.R-project.org/posting- > > > guide.html > > > and provide commented, minimal, self-contained, reproducible code. ______________________________________________ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.