The main issues with this are: 1) ciftiopen and ft_read_cifti return matrices of different dimension, with ft_read_cifti's being larger than what is actually in the file 2) ft_write_cifti puts those added NaNs *into the output file*, such that attempting to use them in wb_command with cifti that "should" match results in an error
Tim On Tue, Oct 25, 2016 at 6:13 PM, Benjamin Risk <[email protected]> wrote: > Right, the NaNs are all in the location of the medial wall, so you can go > back and forth between your representation in ciftiopen and ft_read_cifti > using the code in the previous email. To see this, > > sum(nomedialwall==0) > tabulate(subjLR1.brainstructure(nomedialwall==0)) > > > which returns > > ans = > > 5572 > > Value Count Percent > 1 2796 50.18% > 2 2776 49.82% > > So there are 2796 vertices in the left cortex in the freesurfer 32k space > corresponding to the medial wall and 2776 in the right. Does this seem > correct to you? > > > On Tue, Oct 25, 2016 at 7:04 PM, Glasser, Matthew <[email protected]> > wrote: > >> The issue we have found is that it doesn’t properly represent the >> grayordinates space (it adds NANs). >> >> Peace, >> >> Matt. >> >> From: Benjamin Risk <[email protected]> >> Date: Tuesday, October 25, 2016 at 5:59 PM >> To: Matt Glasser <[email protected]> >> Cc: Timothy Coalson <[email protected]>, "Harms, Michael" <[email protected]>, >> "[email protected]" <[email protected]> >> >> Subject: Re: [HCP-Users] Problem with handling dense FC matrix >> >> In my opinion, there are some advantages to ft_read_cifti. I personally >> have not run into bugs with ft_read_cifti in my applications and am able to >> go back and forth between wb_view and matlab with vertices and voxels in >> the correct ordering. The main benefit of ft_read_cifti is that it reads in >> additional information, for example, reading the rs-fMRI data results in >> >> >> subject=num2str(100206); >> >> subjLR1 = ft_read_cifti(['~/HCP_rs/Subjects/' subject >> '/rfMRI_REST1_LR_Atlas_MSMAll_hp2000_clean.dtseries.nii']); >> subjLR1 >> >> subjLR1 = >> >> dimord: 'pos_time' >> time: [1x1200 double] >> hdr: [1x1 struct] >> unit: 'mm' >> brainstructure: [96854x1 double] >> brainstructurelabel: {1x21 cell} >> dim: [91 109 91] >> pos: [96854x3 double] >> transform: [4x4 double] >> dtseries: [96854x1200 double] >> >> and in particular .brainstructure and .brainstructurelabel can be used to >> extract subcortical regions easily. >> >> In contrast, ciftiopen returns a gifti object containing a single array >> >> >> subjLR1_ciftiopen = ciftiopen(['~/HCP_rs/Subjects/' subject >> '/rfMRI_REST1_LR_Atlas_MSMAll_hp2000_clean.dtseries.nii'],... >> '/home/samsi/Applications2/workbench/bin_linux64/wb_command'); >> >> subjLR1_ciftiopen >> >> subjLR1_ciftiopen = >> >> cdata: [91282x1200 single] >> >> >> I just made this example of how to get the data to match up: >> >> >> subject=num2str(100206); >> >> subjLR1 = ft_read_cifti(['~/HCP_rs/Subjects/' subject >> '/rfMRI_REST1_LR_Atlas_MSMAll_hp2000_clean.dtseries.nii']); >> subjLR1 >> >> % optional: >> subjLR1.dtseries = single(subjLR1.dtseries); >> >> >> subjLR1_ciftiopen = ciftiopen(['~/HCP_rs/Subjects/' subject >> '/rfMRI_REST1_LR_Atlas_MSMAll_hp2000_clean.dtseries.nii'],... >> '/home/samsi/Applications2/workbench/bin_linux64/wb_command'); >> >> subjLR1_ciftiopen >> >> >> nomedialwall = ~isnan(subjLR1.dtseries(:,1)); >> >> test = subjLR1.dtseries(nomedialwall,:); >> size(test) >> >> all(subjLR1_ciftiopen.cdata(:)==test(:)) >> >> norm(subjLR1_ciftiopen.cdata - test,'fro') >> >> >> If people have other thoughts on ft_read_cifti or concerns with anything >> above, maybe we can get a new thread going. >> >> Best, >> Ben >> >> >> On Tue, Oct 25, 2016 at 6:20 PM, Glasser, Matthew <[email protected]> >> wrote: >> >>> ciftiopen uses single precision. I remain puzzled where people are >>> getting the idea to use ft_read_cifti from for MRI data. >>> >>> Peace, >>> >>> Matt. >>> >>> From: <[email protected]> on behalf of Timothy >>> Coalson <[email protected]> >>> Date: Tuesday, October 25, 2016 at 3:55 PM >>> To: "Harms, Michael" <[email protected]> >>> Cc: "[email protected]" <[email protected]> >>> Subject: Re: [HCP-Users] Problem with handling dense FC matrix >>> >>> The ciftiopen option of matlab cifti support should allow you to load >>> the file, though I think it loads as double, and you'd need to convert it >>> to single to reclaim some memory (which comes close to your memory+swap >>> cap, as it needs the double and single versions allocated simultaneously in >>> order to do the conversion). >>> >>> One possibility, if you don't really need a 90k x 90k network, is >>> downsampling or parcellating the dense connectome, see >>> http://www.humanconnectome.org/software/workbench-comman >>> d.php?function=-cifti-parcellate . >>> >>> If you do need the 90k x 90k resolution, and can work in c++ instead of >>> matlab, you could use CiftiLib for reading the file (can read single rows >>> from disk, rather than the entire matrix into memory), or code up a new >>> command for wb_command (same support for cifti, but also support for other >>> file types, access to other algorithms). >>> >>> Tim >>> >>> >>> On Tue, Oct 25, 2016 at 3:21 PM, Harms, Michael <[email protected]> >>> wrote: >>> >>>> >>>> If you’re talking about some sort of community/modularity analysis, >>>> then, no, there aren’t any tools for doing that directly in workbench. >>>> >>>> Maybe Tim C. will have some suggestions. I believe that the dconn is >>>> stored as a single precision float. Have you tried using ‘ciftiopen’? I’m >>>> not sure if that preserves the single precision format of the data, or if >>>> it balloons by a factor of 2 due to conversion to matlab’s default double >>>> precision format. If you can keep it as single precision in matlab, you >>>> might be able to work with it provided Matlab has access to a lot of RAM. >>>> >>>> cheers, >>>> -MH >>>> >>>> -- >>>> Michael Harms, Ph.D. >>>> ----------------------------------------------------------- >>>> Conte Center for the Neuroscience of Mental Disorders >>>> Washington University School of Medicine >>>> Department of Psychiatry, Box 8134 >>>> 660 South Euclid Ave.Tel: 314-747-6173 >>>> St. Louis, MO 63110Email: [email protected] >>>> >>>> From: "GLOMB, KATHARINA" <[email protected]> >>>> Date: Tuesday, October 25, 2016 at 10:35 AM >>>> To: Michael Harms <[email protected]> >>>> Subject: Re: [HCP-Users] Problem with handling dense FC matrix >>>> >>>> Hi Michael, >>>> >>>> wow, thank you for this lightning speed reply! I want to run some >>>> network analysis on this matrix.I don't need the whole matrix, I would just >>>> need the unique entries (off-diagonal upper or lower triangular, >>>> squareform() in Matlab) - maybe that would help? Can workbench help me in >>>> any way to do this? >>>> >>>> Or do you have any other ideas? >>>> >>>> Thank you! >>>> Katharina >>>> >>>> On Tue, Oct 25, 2016 at 5:21 PM, Harms, Michael <[email protected]> >>>> wrote: >>>> >>>>> >>>>> What are you trying to do with the file in workbench? Have you >>>>> reviewed the tutorial, because it includes examples (and a scene to help >>>>> you along) of using that file. >>>>> >>>>> Trying to load/work with that file in matlab is probably going to >>>>> cause loads of pain. It is just too large to load the entire thing into >>>>> matlab. >>>>> >>>>> cheers, >>>>> -MH >>>>> >>>>> -- >>>>> Michael Harms, Ph.D. >>>>> ----------------------------------------------------------- >>>>> Conte Center for the Neuroscience of Mental Disorders >>>>> Washington University School of Medicine >>>>> Department of Psychiatry, Box 8134 >>>>> 660 South Euclid Ave.Tel: 314-747-6173 >>>>> St. Louis, MO 63110Email: [email protected] >>>>> >>>>> From: <[email protected]> on behalf of "GLOMB, >>>>> KATHARINA" <[email protected]> >>>>> Date: Tuesday, October 25, 2016 at 10:15 AM >>>>> To: "[email protected]" <[email protected]> >>>>> Subject: [HCP-Users] Problem with handling dense FC matrix >>>>> >>>>> Hello HCP users, >>>>> >>>>> I am new to HCP so please excuse me if I'm asking something that >>>>> should be obvious. I've tried to find a solution but haven't been >>>>> successful so far. I am trying to use the file >>>>> HCP_S900_820_rfMRI_MSMAll_groupPCA_d4500ROW_zcorr.dconn.nii which to >>>>> my understanding should contain a full FC matrix that is an average of 820 >>>>> subjects. I want to feed this matrix into a Matlab function. Of course, >>>>> this is problematic because of the memory requirements. I have tried to >>>>> read the file using ft_read_cifti from fieldtrip, but Matlab crashes at >>>>> 112 >>>>> GB of memory. So clearly, this is not the way. >>>>> >>>>> I know I am supposed to be able to use workbench for that, but can't >>>>> make it work so far. Can someone point me in the right direction? Is what >>>>> I'm trying to do impossible? >>>>> >>>>> I am using a cluster that can handle up to 132 GB of memory per cpu >>>>> (RAM+swap) and runs Debian. >>>>> >>>>> Thank you!! >>>>> Katharina >>>>> >>>>> -- >>>>> PhD candidate <http://www.cns.upf.edu/katharina> >>>>> Computational Neuroscience Group <http://www.cns.upf.edu/> >>>>> Center for Brain and Cognition <http://cbc.upf.edu/> >>>>> Universitat Pompeu Fabra <http://www.upf.edu/> >>>>> Barcelona, Spain >>>>> >>>>> I am funded by the INDIREA <http://www.indirea.eu/> Marie Curie >>>>> Initial Training Network. >>>>> >>>>> _______________________________________________ >>>>> HCP-Users mailing list >>>>> [email protected] >>>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> The materials in this message are private and may contain Protected >>>>> Healthcare Information or other information of a sensitive nature. If you >>>>> are not the intended recipient, be advised that any unauthorized use, >>>>> disclosure, copying or the taking of any action in reliance on the >>>>> contents >>>>> of this information is strictly prohibited. If you have received this >>>>> email >>>>> in error, please immediately notify the sender via telephone or return >>>>> mail. >>>>> >>>> >>>> >>>> >>>> -- >>>> PhD candidate <http://www.cns.upf.edu/katharina> >>>> Computational Neuroscience Group <http://www.cns.upf.edu/> >>>> Center for Brain and Cognition <http://cbc.upf.edu/> >>>> Universitat Pompeu Fabra <http://www.upf.edu/> >>>> Barcelona, Spain >>>> >>>> I am funded by the INDIREA <http://www.indirea.eu/> Marie Curie >>>> Initial Training Network. >>>> >>>> >>>> ------------------------------ >>>> >>>> The materials in this message are private and may contain Protected >>>> Healthcare Information or other information of a sensitive nature. If you >>>> are not the intended recipient, be advised that any unauthorized use, >>>> disclosure, copying or the taking of any action in reliance on the contents >>>> of this information is strictly prohibited. If you have received this email >>>> in error, please immediately notify the sender via telephone or return >>>> mail. >>>> >>>> _______________________________________________ >>>> HCP-Users mailing list >>>> [email protected] >>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>>> >>> >>> _______________________________________________ >>> HCP-Users mailing list >>> [email protected] >>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>> >>> >>> ------------------------------ >>> >>> The materials in this message are private and may contain Protected >>> Healthcare Information or other information of a sensitive nature. If you >>> are not the intended recipient, be advised that any unauthorized use, >>> disclosure, copying or the taking of any action in reliance on the contents >>> of this information is strictly prohibited. If you have received this email >>> in error, please immediately notify the sender via telephone or return mail. >>> >>> _______________________________________________ >>> HCP-Users mailing list >>> [email protected] >>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>> >> >> >> ------------------------------ >> >> The materials in this message are private and may contain Protected >> Healthcare Information or other information of a sensitive nature. If you >> are not the intended recipient, be advised that any unauthorized use, >> disclosure, copying or the taking of any action in reliance on the contents >> of this information is strictly prohibited. If you have received this email >> in error, please immediately notify the sender via telephone or return mail. >> > > _______________________________________________ HCP-Users mailing list [email protected] http://lists.humanconnectome.org/mailman/listinfo/hcp-users
