This makes more sense! Sorry I missed your mention of fisher-z transform So I would apply tanh to each element to revert back to regular correlation coefficients
Thank you for your help! Best, Sang On Mon, May 15, 2017 at 5:57 PM Timothy Coalson <tsc...@mst.edu> wrote: > After the fisher-z transform, you can have values greater than 1, see the > graph on the right: > > https://en.wikipedia.org/wiki/Fisher_transformation > > This is why the "correct" answer for the diagonal is infinity for the > "zcorr" file. > > Tim > > > On Mon, May 15, 2017 at 7:51 PM, Sang-Yun Oh <san...@gmail.com> wrote: > >> I am also finding that some off-diagonal elements in this matrix are also >> greater than 1 indicating this matrix is not a correlation matrix. >> >> In [5]: img >> Out[5]: >> memmap([[ 8.66434002e+00, 1.96847185e-01, 1.66294336e-01, ..., >> 1.01449557e-01, 7.45474100e-02, 1.15624115e-01], >> [ 1.96847185e-01, inf, 3.36383432e-01, ..., >> -5.70017472e-03, -5.49946353e-02, 3.72834280e-02], >> [ 1.66294336e-01, 3.36383432e-01, inf, ..., >> -4.45242636e-02, -6.07097335e-02, -1.51601573e-02], >> ..., >> [ 1.01449557e-01, -5.70017472e-03, -4.45242636e-02, ..., >> inf, 1.91883039e+00, 9.20160294e-01], >> [ 7.45474100e-02, -5.49946353e-02, -6.07097335e-02, ..., >> * 1.91883111e+00*, 8.31776619e+00, 8.82132888e-01], >> [ 1.15624115e-01, 3.72833721e-02, -1.51601573e-02, ..., >> 9.20160294e-01, 8.82132888e-01, 8.66434002e+00]], >> dtype=float32) >> >> Any insight would be appreciated >> >> Thanks, >> Sang >> >> On Mon, May 15, 2017 at 1:13 PM Sang-Yun Oh <san...@gmail.com> wrote: >> >>> Thank you for the response. >>> >>> I am, too, confused by some being non-zero finite values, and others >>> being infinities. >>> >>> Before computing a correlation matrix, if standardized by subtracting >>> the mean and scaling by variance, all diagonal elements should be exactly 1. >>> >>> What I am concerned about is how the whole matrix was computed, since a >>> fundamental characteristic of correlation matrix is not satisfied >>> >>> Best, >>> Sang >>> >>> On Mon, May 15, 2017 at 11:33 AM Timothy Coalson <tsc...@mst.edu> wrote: >>> >>>> Per the name "zcorr", the correlation values have been z-transformed >>>> (fisher's small z transform). I am somewhat confused as to why some >>>> elements in the diagonal are not infinite. The "true" value of applying >>>> this transform would be infinite on the entire diagonal, as arctanh(1) is >>>> infinite. I am guessing this result was generated in matlab, as wb_command >>>> actually prevents infinities when using the z transform, putting a cap on >>>> the correlation (when not using z-transform, it shows correlations of 1 as >>>> expected). >>>> >>>> Whatever analysis you do with correlation matrices like this should >>>> ignore the diagonal anyway, since it is correlation to itself. >>>> >>>> Tim >>>> >>>> >>>> On Mon, May 15, 2017 at 3:57 AM, Sang-Yun Oh <san...@gmail.com> wrote: >>>> >>>>> I downloaded group average functional correlation >>>>> file: HCP_S900_820_rfMRI_MSMAll_groupPCA_d4500ROW_zcorr.dconn.nii >>>>> >>>>> Some diagonal elements of the square matrix (91282x91282) are infinite >>>>> (Please see below). >>>>> >>>>> I want to use this matrix in ananalysis; however, I am not sure how to >>>>> understand or deal with infinite diagonal values. >>>>> >>>>> I appreciate any insight >>>>> >>>>> Thanks, >>>>> Sang >>>>> >>>>> ====================== >>>>> >>>>> In [1]: import nibabel >>>>> >>>>> In [2]: asdf = >>>>> nibabel.load('HCP_S900_820_rfMRI_MSMAll_groupPCA_d4500ROW_zcorr.dconn.nii') >>>>> >>>>> In [3]: img = asdf.get_data() >>>>> >>>>> In [4]: img.shape >>>>> Out[4]: (1, 1, 1, 1, 91282, 91282) >>>>> >>>>> In [5]: S = img[0,0,0,0,:,:] >>>>> >>>>> In [6]: S >>>>> Out[6]: >>>>> memmap([[ 8.66434002e+00, 1.96847185e-01, 1.66294336e-01, ..., >>>>> 1.01449557e-01, 7.45474100e-02, 1.15624115e-01], >>>>> [ 1.96847185e-01, inf, 3.36383432e-01, ..., >>>>> -5.70017472e-03, -5.49946353e-02, 3.72834280e-02], >>>>> [ 1.66294336e-01, 3.36383432e-01, inf, ..., >>>>> -4.45242636e-02, -6.07097335e-02, -1.51601573e-02], >>>>> ..., >>>>> [ 1.01449557e-01, -5.70017472e-03, -4.45242636e-02, ..., >>>>> inf, 1.91883039e+00, 9.20160294e-01], >>>>> [ 7.45474100e-02, -5.49946353e-02, -6.07097335e-02, ..., >>>>> 1.91883111e+00, 8.31776619e+00, 8.82132888e-01], >>>>> [ 1.15624115e-01, 3.72833721e-02, -1.51601573e-02, ..., >>>>> 9.20160294e-01, 8.82132888e-01, 8.66434002e+00]], >>>>> dtype=float32) >>>>> >>>>> In [7]: S.diagonal() >>>>> Out[7]: >>>>> memmap([ 8.66434002, inf, inf, ..., inf, >>>>> 8.31776619, 8.66434002], dtype=float32) >>>>> >>>>> >>>>> _______________________________________________ >>>>> HCP-Users mailing list >>>>> HCP-Users@humanconnectome.org >>>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>>>> >>>> >>>> > _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
