Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Thanks for the many pointers. At this stage, although R is installed on my computer, I would hesitate learning R just to run the Nathan Mantel test. Can someone give me a helping hand in defining a spatial coordination index? Using - in the first place - the random xyz coordinates below that we had been using before for the nearest neighbour distribution which had worked out very well in accordance theoretical predictions. Apllication of covariance/correlance analysis turned out to be useless, because this investigates a search into linear dependencies of the x, y, z vectors and this is not of interest. Now, with the numbers generated below, the spatial coordination index looking for clusters and holes in 3d space should come out near zero. Somebody trying to run the numbers? For an xy projection, see < https://www.dropbox.com/s/0sv1ak9zwk5f8rv/16%2C867%20points%20in%203d%20spac e.pptx?dl=0>. Heinz // START OF CODE FOR RANDOM x, y, z NUMBER GENERATION n=16867; r=25046; radius = r*grand(n,1,'def').^(1/3); phi = 2*%pi*grand(n,1, 'def'); costheta = 1 - 2*grand(n,1, 'def'); radsintheta = radius.*sin(acos(costheta)); x=radsintheta.*cos(phi); y=radsintheta.*sin(phi); z=radius.*costheta; // END OF CODE -Original Message- From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Stéphane Mottelet Sent: 09 February 2018 09:47 To: Users mailing list for Scilab Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? Hello, There seems to be some statistical tests related to this, namely Morans or Mantel tests : https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-how-can-i-detecta ddress-spatial-autocorrelation-in-my-data/ http://www.petrkeil.com/?p=1050 hth S. Le 08/02/2018 à 20:16, Heinz a écrit : > Sorry, I am lost. > > I have 10,000 xyz data and want to know, if there is some regularity > in them or if they are more or less random. The concept of a mask does > not come up at all... > Best greetings > Heinz > > What do you suggest should be used here? > > -Original Message- > From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of > Samuel Gougeon > Sent: 08 February 2018 19:19 > To: Users mailing list for Scilab > Subject: Re: [Scilab-users] spatial correlation coefficient: Is the > MATLAB corrcoef function doing the job? > > Hello Heinz, > > Le 07/02/2018 à 23:13, Heinz a écrit : >> Friends: >> >> I need to compute the spatial correlation coefficient of my x,y,z >> data in > 3d. >> Question 1: Is the MATLAB corrcoef function doing the job? >> Question 2: How do I get corrcoef running in SciLab or C? > Do you mean the autocorrelation of a data(x,y,z) sampled distribution? > Or the intercorrelation of data(x,y,z) with a mask(x,y,z) function > sampled on the same grid? > Or anything else? > > Samuel > > > ___ > users mailing list > users@lists.scilab.org > http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
You may want to study this article: https://pdfs.semanticscholar.org/574f/8f7347a258f6431fb8316716ca51b9b7eacb.pdf it should give you a clue or two of what you may want to implement. -Original Message- From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Heinz Sent: Friday, February 09, 2018 7:44 PM To: 'Users mailing list for Scilab' Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? 1 The nearest neighbour distribution of N points in a 3d-Volume with radius R is given by W(r) = 1 - exp[ - N (r/r)^3] and this is a Weibull distribution and not a Poisson distribution. The WIKI article, while correct, is a typical case where high level statisticians are trying to make simple things look very complicated. 2 Comparing the experimental distribution with the above prediction is made complicated as the manufactured object has small hard spheres that push each other apart in the manufacturing process and the Weibull is modified anyway. 3 I am sill searching for a 3d spatial characterization coefficient that differentiates between purely random arrangements and gradual regular arrangements. Heinz -Original Message- From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Rafael Guerra Sent: 09 February 2018 10:17 To: Users mailing list for Scilab Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? Q: "I have 10,000 xyz data and want to know, if there is some regularity in them or if they are more or less random" A: look for tests of spatial randomness (https://en.wikipedia.org/wiki/Complete_spatial_randomness ) . If The experimental nearest neighbor distribution obtained previously can be compared to the expected Poisson distribution. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
1 The nearest neighbour distribution of N points in a 3d-Volume with radius R is given by W(r) = 1 - exp[ - N (r/r)^3] and this is a Weibull distribution and not a Poisson distribution. The WIKI article, while correct, is a typical case where high level statisticians are trying to make simple things look very complicated. 2 Comparing the experimental distribution with the above prediction is made complicated as the manufactured object has small hard spheres that push each other apart in the manufacturing process and the Weibull is modified anyway. 3 I am sill searching for a 3d spatial characterization coefficient that differentiates between purely random arrangements and gradual regular arrangements. Heinz -Original Message- From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Rafael Guerra Sent: 09 February 2018 10:17 To: Users mailing list for Scilab Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? Q: "I have 10,000 xyz data and want to know, if there is some regularity in them or if they are more or less random" A: look for tests of spatial randomness (https://en.wikipedia.org/wiki/Complete_spatial_randomness ) . If The experimental nearest neighbor distribution obtained previously can be compared to the expected Poisson distribution. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Q: "I have 10,000 xyz data and want to know, if there is some regularity in them or if they are more or less random" A: look for tests of spatial randomness (https://en.wikipedia.org/wiki/Complete_spatial_randomness ) . If The experimental nearest neighbor distribution obtained previously can be compared to the expected Poisson distribution. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Hello, There seems to be some statistical tests related to this, namely Moran’s or Mantel tests : https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-how-can-i-detectaddress-spatial-autocorrelation-in-my-data/ http://www.petrkeil.com/?p=1050 hth S. Le 08/02/2018 à 20:16, Heinz a écrit : Sorry, I am lost. I have 10,000 xyz data and want to know, if there is some regularity in them or if they are more or less random. The concept of a mask does not come up at all... Best greetings Heinz What do you suggest should be used here? -Original Message- From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Samuel Gougeon Sent: 08 February 2018 19:19 To: Users mailing list for Scilab Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? Hello Heinz, Le 07/02/2018 à 23:13, Heinz a écrit : Friends: I need to compute the spatial correlation coefficient of my x,y,z data in 3d. Question 1: Is the MATLAB corrcoef function doing the job? Question 2: How do I get corrcoef running in SciLab or C? Do you mean the autocorrelation of a data(x,y,z) sampled distribution? Or the intercorrelation of data(x,y,z) with a mask(x,y,z) function sampled on the same grid? Or anything else? Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Le 08/02/2018 à 20:16, Heinz a écrit : Sorry, I am lost. I have 10,000 xyz data and want to know, if there is some regularity in them or if they are more or less random. So you need and are speaking about the 3D autocorrelation of data(x,y,z). Usually we compute it through a 3D convolution, that is easy to compute through the FFT, noting that for an autocorrelation we just flip one of both data array along x,y, and z before computing its FFT and going on with the algorithm. Finally we use the inverse FFT to come back to the direct x,y,z space. You may find good references on the web about the keyword 3D autocorrelation. In scilab, you will mainly need fft(A,sign,dims,incr[,option] ) since xcorr(), conv() conv2() convol2d() .. work only in 1D or 2D (with too often duplicates to do the same thing, without "simple" extension to do more...) Best regards Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Rafael Guerra Sent: 08 February 2018 18:55 To: Users mailing list for Scilab Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? Hi Heinz, Scilab computes the covariance matrix and from which the correlation matrix can be obtained using formula in https://en.wikipedia.org/wiki/Covariance_matrix Check implementation below: //START OF CODE // https://en.wikipedia.org/wiki/Covariance_matrix function Y=corrmatrix(M) C = cov(M); // covariance matrix D = sqrt(diag(C)); // standard deviations D = inv(diag(D)); Y = D*C*D; // correlation matrix endfunction M = grand(9,3,"def") M(:,2) = M(:,1)*2; Y = corrmatrix(M); disp(M,"M") disp(Y,"Y") // END OF CODE For random generated xyz positions, I get the matrix below and this looks fine with me: 1.000 -0.009 -0.003 -0.009 1.000 -0.001 -0.003 -0.001 1.000 But this here is a manufactured object and every singly xyz-value is obtained from X-ray tomography https://www.dropbox.com/s/87osn38agn8jfzo/Measured%2016867%20data%20points%2 0in%203d.png?dl=0 and looks much more regular than the Monte-Carlo. However, the correlation analysis suggested here, gives much the same numbers. 1.000 -0.009 0.008 -0.009 1.000 -0.001 0.008 -0.001 1.000 Perhaps, I have asked the wrong question: -->what I need is one single figures that summarizes the spatial correlation. Heinz ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Sorry, I am lost. I have 10,000 xyz data and want to know, if there is some regularity in them or if they are more or less random. The concept of a mask does not come up at all... Best greetings Heinz What do you suggest should be used here? -Original Message- From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Samuel Gougeon Sent: 08 February 2018 19:19 To: Users mailing list for Scilab Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? Hello Heinz, Le 07/02/2018 à 23:13, Heinz a écrit : > Friends: > > I need to compute the spatial correlation coefficient of my x,y,z data in 3d. > > Question 1: Is the MATLAB corrcoef function doing the job? > Question 2: How do I get corrcoef running in SciLab or C? Do you mean the autocorrelation of a data(x,y,z) sampled distribution? Or the intercorrelation of data(x,y,z) with a mask(x,y,z) function sampled on the same grid? Or anything else? Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Thanks. While not giving me ONE number, the analysis shows that my random generated xyz positions are uncorrelated as they should. Can one estimate the error bars for the xy xz and yz correlations? Now, I can use this approach for actual measurements in real objects. Heinz //START function Y=corrmatrix(M) C = cov(M); // covariance matrix D = sqrt(diag(C)); // standard deviations D = inv(diag(D)); Y = D*C*D; // correlation matrix endfunction n=1; r=23; radius = r*grand(n,1,'def').^(1/3); phi = 2*%pi*grand(n,1, 'def'); costheta = 1 - 2*grand(n,1, 'def'); radsintheta = radius.*sin(acos(costheta)); X = [radsintheta.*cos(phi),radsintheta.*sin(phi), radius.*costheta]; Y = corrmatrix(X) 1. 0.0058073 0.0078778 0.0058073 1. 0.0014375 0.0078778 0.0014375 1. //END From: users [mailto:users-boun...@lists.scilab.org] On Behalf Of Rafael Guerra Sent: 08 February 2018 18:55 To: Users mailing list for Scilab Subject: Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job? Hi Heinz, Scilab computes the covariance matrix and from which the correlation matrix can be obtained using formula in https://en.wikipedia.org/wiki/Covariance_matrix Check implementation below: //START OF CODE // https://en.wikipedia.org/wiki/Covariance_matrix function Y=corrmatrix(M) C = cov(M); // covariance matrix D = sqrt(diag(C)); // standard deviations D = inv(diag(D)); Y = D*C*D; // correlation matrix endfunction M = grand(9,3,"def") M(:,2) = M(:,1)*2; Y = corrmatrix(M); disp(M,"M") disp(Y,"Y") // END OF CODE Regards, Rafael ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Yes, a precise math formula would disambiguate the denomination. S. > Le 8 févr. 2018 à 19:18, Samuel Gougeon a écrit : > > Hello Heinz, > >> Le 07/02/2018 à 23:13, Heinz a écrit : >> Friends: >> >> I need to compute the spatial correlation coefficient of my x,y,z data in 3d. >> >> Question 1: Is the MATLAB corrcoef function doing the job? >> Question 2: How do I get corrcoef running in SciLab or C? > > Do you mean the autocorrelation of a data(x,y,z) sampled distribution? > Or the intercorrelation of data(x,y,z) with a mask(x,y,z) function sampled on > the same grid? > Or anything else? > > Samuel > > ___ > users mailing list > users@lists.scilab.org > http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Hello Heinz, Le 07/02/2018 à 23:13, Heinz a écrit : Friends: I need to compute the spatial correlation coefficient of my x,y,z data in 3d. Question 1: Is the MATLAB corrcoef function doing the job? Question 2: How do I get corrcoef running in SciLab or C? Do you mean the autocorrelation of a data(x,y,z) sampled distribution? Or the intercorrelation of data(x,y,z) with a mask(x,y,z) function sampled on the same grid? Or anything else? Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] spatial correlation coefficient: Is the MATLAB corrcoef function doing the job?
Hi Heinz, Scilab computes the covariance matrix and from which the correlation matrix can be obtained using formula in https://en.wikipedia.org/wiki/Covariance_matrix Check implementation below: //START OF CODE // https://en.wikipedia.org/wiki/Covariance_matrix function Y=corrmatrix(M) C = cov(M); // covariance matrix D = sqrt(diag(C)); // standard deviations D = inv(diag(D)); Y = D*C*D; // correlation matrix endfunction M = grand(9,3,"def") M(:,2) = M(:,1)*2; Y = corrmatrix(M); disp(M,"M") disp(Y,"Y") // END OF CODE Regards, Rafael ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
