If you are using -cifti-correlation, there is a -mem-limit option for this purpose, so there isn't a required minimum memory to do it (even in matlab where everything has to be in memory, the 90k x 4.8k timeseries input pales in comparison to the 90k x 90k output). If you are doing everything in matlab, then the averaging of two 90k x 90k dconns is going to require more memory than any reasonable concatenated correlation.
The -cifti-average command should use almost no memory regardless of file size, as long as you don't overwrite one of the inputs with the output. Tim On Tue, Nov 24, 2015 at 2:23 PM, Greg Burgess <[email protected]> wrote: > I suggested below that Joelle could average Fisher’s z-transformed > correlation coefficients (derived from each run within-subject), or treat > the multiple runs as within-subjects repeated measures. > > The idea was that computing correlations between timeseries with 4800 time > points will take four times as much RAM as using only 1200 time points. For > folks with limited RAM, averaging the correlation estimates may be a more > feasible option. > > --Greg > > ____________________________________________________________________ > Greg Burgess, Ph.D. > Staff Scientist, Human Connectome Project > Washington University School of Medicine > Department of Anatomy and Neurobiology > Phone: 314-362-7864 > Email: [email protected] > > > On Nov 24, 2015, at 1:08 PM, Stephen Smith <[email protected]> wrote: > > > > I think maybe we need to be explicit about exactly what we're talking > about averaging? > > Cheers. > > > > -------------------- > > Stephen M. Smith, Professor of Biomedical Engineering > > Head of Analysis, Oxford University FMRIB Centre > > > > FMRIB, JR Hospital, Headington, > > Oxford. OX3 9 DU, UK > > +44 (0) 1865 222726 (fax 222717) > > [email protected] > > http://www.fmrib.ox.ac.uk/~steve > > ---------------------- > > > >> On 24 Nov 2015, at 19:03, Greg Burgess <[email protected]> wrote: > >> > >> It’s less RAM-intensive since you only need to load one timeseries at a > time. > >> > >> --Greg > >> > >> ____________________________________________________________________ > >> Greg Burgess, Ph.D. > >> Staff Scientist, Human Connectome Project > >> Washington University School of Medicine > >> Department of Anatomy and Neurobiology > >> Phone: 314-362-7864 > >> Email: [email protected] > >> > >>> On Nov 24, 2015, at 12:29 PM, Glasser, Matthew <[email protected]> > wrote: > >>> > >>> I'm not sure what benefit you'd get from averaging the FCs across runs > within a subject. That just sounds more computationally intensive. > >>> > >>> Peace, > >>> > >>> Matt. > >>> > >>> > >>> From: Joelle Zimmermann <[email protected]> > >>> Sent: Tuesday, November 24, 2015 11:55 AM > >>> To: Glasser, Matthew > >>> Cc: Greg Burgess; Elam, Jennifer; [email protected] > >>> Subject: Re: [HCP-Users] Phase Encoding left-to-right and right-to-left > >>> > >>> Hi Matt, > >>> > >>> Glad you do point that out, because I was previously looking at the > Resting State fMRI 1 Preprocessed, but the Resting State fMRI FIX-Denoised > (Compact) is readily available. So I guess for that all I'll need to do is > demean and variance normalize, and/or average the two FCs. > >>> > >>> Thanks, > >>> Joelle > >>> > >>> On Tue, Nov 24, 2015 at 12:19 PM, Glasser, Matthew <[email protected]> > wrote: > >>> Indeed I was assuming you were using FIX cleaned data. I wouldn't > recommend not using FIX cleaned data unless you are testing other clean up > approaches. > >>> > >>> Peace, > >>> > >>> Matt. > >>> ________________________________________ > >>> From: Joelle Zimmermann [[email protected]] > >>> Sent: Tuesday, November 24, 2015 11:14 AM > >>> To: Greg Burgess > >>> Cc: Glasser, Matthew; Elam, Jennifer; [email protected] > >>> Subject: Re: [HCP-Users] Phase Encoding left-to-right and right-to-left > >>> > >>> Hi Greg, > >>> > >>> Thanks for your response. Indeed, I was considering that myself, to > compute the FCs separately and average the LR and RL. > >>> > >>> Thanks, > >>> Joelle > >>> > >>> On Tue, Nov 24, 2015 at 11:56 AM, Greg Burgess <[email protected] > <mailto:[email protected]>> wrote: > >>> Hi Joelle, > >>> > >>> In addition to demeaning and possibly variance normalization, it is > probably a good idea to detrend each run separately using a linear detrend > or a high pass filter before concatenation. (FIX-preprocessed data already > includes a 2000s high pass filter.) > >>> > >>> Another option that is not described on the wiki (yet) is to compute > correlations separately for each run, and then average the Fisher’s > z-transformed correlation coefficients, or treat the multiple runs as > within-subjects repeated measures. > >>> > >>> --Greg > >>> > >>> ____________________________________________________________________ > >>> Greg Burgess, Ph.D. > >>> Staff Scientist, Human Connectome Project > >>> Washington University School of Medicine > >>> Department of Anatomy and Neurobiology > >>> Phone: 314-362-7864<tel:314-362-7864> > >>> Email: [email protected]<mailto:[email protected]> > >>> > >>>> On Nov 23, 2015, at 4:40 PM, Glasser, Matthew <[email protected] > <mailto:[email protected]>> wrote: > >>>> > >>>> It doesn’t matter what order you concatenate the data in, but I would > not recommend only analyzing the data of one phase encoding direction. > >>>> > >>>> Peace, > >>>> > >>>> Matt. > >>>> > >>>> From: <[email protected]<mailto: > [email protected]>> on behalf of Joelle Zimmermann < > [email protected]<mailto:[email protected]>> > >>>> Date: Monday, November 23, 2015 at 12:48 PM > >>>> To: "Elam, Jennifer" <[email protected]<mailto:[email protected]>> > >>>> Cc: "[email protected]<mailto: > [email protected]>" <[email protected]<mailto: > [email protected]>> > >>>> Subject: Re: [HCP-Users] Phase Encoding left-to-right and > right-to-left > >>>> > >>>> Hi Jennifer and Matt, > >>>> > >>>> Thanks for your help. I have a few clarification questions below: > >>>> Does it matter in which order I concatenate the LR and the RL .nii's? > My ultimate goal is to create a functional connectivity matrix from the > time series. > >>>> #3 in the link you sent describes that there are 4 runs per subject. > Is this the REST 1, and REST 2, each with LR and RL phase encoding > directions? > >>>> Would using only one phase encoding direction (i.e. do analysis on > LR) expect to effect the results? > >>>> > >>>> Thanks, > >>>> Joelle > >>>> > >>>>> On Mon, Nov 23, 2015 at 1:17 PM, Jennifer Elam <[email protected] > <mailto:[email protected]>> wrote: > >>>>> #3 on > https://wiki.humanconnectome.org/display/PublicData/HCP+Users+FAQ may be > of help. > >>>>> > >>>>> Best, > >>>>> Jenn > >>>>> > >>>>> Jennifer Elam, Ph.D. > >>>>> Outreach Coordinator, Human Connectome Project > >>>>> Washington University School of Medicine > >>>>> Department of Anatomy and Neurobiology, Box 8108 > >>>>> 660 South Euclid Avenue > >>>>> St. Louis, MO 63110 > >>>>> 314-362-9387<tel:314-362-9387> > >>>>> [email protected]<mailto:[email protected]> > >>>>> www.humanconnectome.org<http://www.humanconnectome.org> > >>>>> > >>>>> From:[email protected]<mailto: > from%[email protected]> [mailto: > [email protected]<mailto: > [email protected]>] On Behalf Of Glasser, Matthew > >>>>> Sent: Monday, November 23, 2015 12:16 PM > >>>>> To: Joelle Zimmermann; [email protected]<mailto: > [email protected]> > >>>>> Subject: Re: [HCP-Users] Phase Encoding left-to-right and > right-to-left > >>>>> > >>>>> > >>>>> > >>>>> Usually you concatenate them temporally after demeaning (and perhaps > variance normalizing). > >>>>> > >>>>> > >>>>> > >>>>> Peace, > >>>>> > >>>>> > >>>>> > >>>>> Matt. > >>>>> > >>>>> > >>>>> > >>>>> From:[email protected]<mailto: > from%[email protected]> < > [email protected]<mailto: > [email protected]>> on behalf of Joelle Zimmermann < > [email protected]<mailto:[email protected]>> > >>>>> Sent: Monday, November 23, 2015 11:33 AM > >>>>> To: [email protected]<mailto: > [email protected]> > >>>>> Subject: [HCP-Users] Phase Encoding left-to-right and right-to-left > >>>>> > >>>>> > >>>>> > >>>>> Hi all, > >>>>> > >>>>> > >>>>> > >>>>> Would anyone be able to explain a bit more about the phase-encoding > directions LR and RL for the (preprocessed) REST1 session data from 500 > subjects +MEG2? I understand that LR is left to right and RL is right to > left. > >>>>> > >>>>> > >>>>> > >>>>> I'm wondering, are these meant to be somehow combined, or is only > one of these typically chosen? > >>>>> > >>>>> > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Joelle > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > http://www.humanconnectome.org/documentation/Q1/data-in-this-release.html > >>>>> > >>>>> Q1 Data Release: About the Dataset | Human Connectome Project > >>>>> 76 healthy adult subjects in the age range 22 – 35 participated in > the first quarter of data collection. These include 68 subjects with data > from all or nearly all ... > >>>>> Read more... > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> HCP-Users mailing list > >>>>> [email protected]<mailto:[email protected]> > >>>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > >>>>> > >>>>> _______________________________________________ > >>>>> HCP-Users mailing list > >>>>> [email protected]<mailto:[email protected]> > >>>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > >>>> > >>>> _______________________________________________ > >>>> HCP-Users mailing list > >>>> [email protected]<mailto:[email protected]> > >>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > >>>> _______________________________________________ > >>>> HCP-Users mailing list > >>>> [email protected]<mailto:[email protected]> > >>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > >>> > >>> > >>> > >>> ________________________________ > >>> The materials in this message are private and may contain Protected > Healthcare Information or other information of a sensitive nature. If you > are not the intended recipient, be advised that any unauthorized use, > disclosure, copying or the taking of any action in reliance on the contents > of this information is strictly prohibited. If you have received this email > in error, please immediately notify the sender via telephone or return mail. > >>> > >>> > >>> The materials in this message are private and may contain Protected > Healthcare Information or other information of a sensitive nature. If you > are not the intended recipient, be advised that any unauthorized use, > disclosure, copying or the taking of any action in reliance on the contents > of this information is strictly prohibited. If you have received this email > in error, please immediately notify the sender via telephone or return mail. > >> > >> > >> _______________________________________________ > >> HCP-Users mailing list > >> [email protected] > >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > > > _______________________________________________ > HCP-Users mailing list > [email protected] > http://lists.humanconnectome.org/mailman/listinfo/hcp-users > _______________________________________________ HCP-Users mailing list [email protected] http://lists.humanconnectome.org/mailman/listinfo/hcp-users
