Hi Matthew,
Thanks a lot for your response, I think I am understanding this a bit more.
I am currently using an algorithm coming from fluid mechanics called
Dynamic Mode Decomposition (
http://www.sciencedirect.com/science/article/pii/S0165027015003829). The
algorithm decomposes the dtseries into a sum of spatially oscillating
modes. When I use the data from MSMAll_hp2000_clean.dtseries.nii, the
algorithm is not able to decompose the signal. However, the pilot data (and
only the pilot data, not the ica, normalized, or the raw) seems to
decompose and I get spatial modes that are similar to functional
connectivity networks seen in previous literature, as well as a
distribution of frequencies that seem to be biological plausible. I think
most of this is due to DMD being very sensitive, and if its off (which I am
not quite sure how to define/characterize) the algorithm just gives me
junk.
Now, I am assuming (maybe incorrectly) that the middle paragraph in your
last response, is what you used to process the pilot data. The mean
centering and the normalization makes sense to me, but I am a bit confused
on what "multiplying the bias field back into the data" refers to. I am
confused what the bias field is (experimentally). Dividing by the
variance definitely throws in a lot of noise (due to the fact that
neighboring voxels probably should have similar std but since we have only
limited data they are not).
But I think, I would like to process the raw data (
MSMAll_hp2000_clean.dtseries.nii) with this method and see if my algorithm
can decompose it. Thanks a lot for your help,
Cheers
Amrit
On Mon, Aug 15, 2016 at 9:27 PM, Glasser, Matthew <glass...@wustl.edu>
wrote:
> Well I processed the pilot data and have said that it is not recommended
> that you use that for analysis. The recommended starting data is the
> ICA+FIX cleaned data which has been registered with MSMAll and is named
> something like this:
>
> ${StudyFolder}/${Subject}/MNINonLinear/Results/${
> fMRIName}/${fMRIName}_Atlas_MSMAll_hp2000_clean.dtseries.nii
>
> If you are interested in looking at dense timeseries movies or
> concatenating across runs, you will want to remove the mean image from the
> data. Next you will need to decide whether you want to variance normalize
> the data (if you do, you will want to multiply the bias field back into the
> data and divide by the variance normalization map—these files are included
> in the same package as the above). Variance normalized data has an equal
> chance of false positives across space, but isn’t interpretable in terms of
> %BOLD deviation across space. The included bias field correction isn’t
> quite optimal and we haven’t had a chance to create the data that enables a
> better bias field correction.
>
> It is indeed true that if you reconstruct the data from the signal ICA
> components that a given run has, it will look a lot smoother spatially and
> temporally, because you will have thrown out all of the unstructured
> noise. This might be okay for individual subject analysis, but you
> wouldn’t want to do this for group analysis (because you would not be able
> to get the benefits of cross-subject averaging for bringing out weaker
> components). I’ve ended up doing something different which we call
> “Wishart Rolloff” which is described in this document:
>
> http://www.humanconnectome.org/documentation/mound-and-moat-effect.html
>
> This makes a bit fewer assumptions than does the ICA-based reconstruction
> approach, and so I have moved away from the ICA-recon approach. In any
> case, I don’t really understand exactly what you are trying to do with the
> data…
>
> Peace,
>
> Matt.
>
> From: Amrit Kashyap <amrit...@gmail.com>
> Date: Monday, August 15, 2016 at 10:35 AM
> To: Matt Glasser <glass...@wustl.edu>
> Cc: "Elam, Jennifer" <e...@wustl.edu>, "hcp-users@humanconnectome.org" <
> hcp-users@humanconnectome.org>
>
> Subject: Re: [HCP-Users] Workbench Tutorial Data
>
>
> HCPfilterdNorm.mpg
> <https://drive.google.com/file/d/0B21oKdbljNLvaThnSy11anFqQzA/view?usp=drive_web>
>
> HCPicacleaned.mpg
> <https://drive.google.com/file/d/0B21oKdbljNLvOE9ISlVLdGt3cmc/view?usp=drive_web>
>
> HCPMSALLraw.mpg
> <https://drive.google.com/file/d/0B21oKdbljNLveUE1SzNHaFZrQzg/view?usp=drive_web>
>
> HCPpilot.mpg
> <https://drive.google.com/file/d/0B21oKdbljNLvYTlxR1dXbjlXY2M/view?usp=drive_web>
> Hey HCP,
>
> I have downloaded the raw data that you have suggested and I tried many
> techniques unsuccessfully (filtering, smoothing, normalizing, ... etc) to
> try to transform the data into the signal that I have observed in the scene
> 4 data. I have included 4 files below to show the difference between the
> HCP pilot processing vs the raw data and vs my own processing.
>
> I believe that this signal (HCP pilot) is probably closer to the true
> signal that we are interested in, because it seems like the dynamics of the
> spatio-temporal pattern has normalized correctly over the entire brain,
> resulting in a smooth signal that transitions from one region to another.
> The raw signal (MSMAII in this case but ICA+fix is very similar) is not
> normalized across the brain space and shows most of the activity clustered
> around regions that inherently have more activity. Normalizing and
> filtering results in a signal that looks similar to the HCP pilot, but
> contains large amounts of spatial noise that is not seen in the HCP pilot
> data. I believe that probably the HCP pilot has been spatially filtered but
> I am not sure. I tried projecting the data into the 300 ICA components as
> well, and then reconstructing the signal from these components to clean the
> signal and it results in the fourth file which looks interesting but does
> not have the smooth dynamics seen from the pilot data.
>
> I think that someone has done an amazing job in post-processing the HCP
> pilot data and their work is a great interest for the scientific community.
> The last few weeks I have attempted to guess what has been done to the
> signal and try to apply it myself but this is not getting me very far. If
> you have any documentation about the pilot study or the code that generated
> the processing for the HCP pilot data I would be incredibly grateful since
> I am struggling to reproduce its results. For my present work, I am trying
> to extend the spatial functional connectivity maps into a temporal domain.
> For this I need very smooth signals both spatially as well as temporally
> and the HCP pilot data gives me promising results in decomposing the
> dynamics.
>
> Thanks so much for all the help you have already provided and appologize
> for the long email,
> Cheers
> Amrit
>
>
>
>
> On Wed, Jul 27, 2016 at 9:46 AM, Glasser, Matthew <glass...@wustl.edu>
> wrote:
>
>> ICA+FIX for all usecases, but otherwise it depends on what you are doing.
>>
>>
>> Peace,
>>
>>
>> Matt.
>> ------------------------------
>> *From:* Amrit Kashyap <amrit...@gmail.com>
>> *Sent:* Wednesday, July 27, 2016 8:20:51 AM
>> *To:* Glasser, Matthew
>> *Cc:* Elam, Jennifer; hcp-users@humanconnectome.org
>>
>> *Subject:* Re: [HCP-Users] Workbench Tutorial Data
>>
>> Awesome, thanks a bunch I will try it out. Just out of curiosity what
>> preprocessing would you recommend to clean up the signal (filtering,
>> interpolation, downsampling)?
>> Cheers
>> Amrit
>>
>> On Tue, Jul 26, 2016 at 1:50 PM, Glasser, Matthew <glass...@wustl.edu>
>> wrote:
>>
>>> There is a wb_command function that does this, wb_command
>>> -cifti-smoothing. We don’t recommend smoothing as a general preprocessing
>>> practice however.
>>>
>>> Peace,
>>>
>>> Matt.
>>>
>>> From: Amrit Kashyap <amrit...@gmail.com>
>>> Date: Tuesday, July 26, 2016 at 12:47 PM
>>> To: "Elam, Jennifer" <e...@wustl.edu>
>>> Cc: Matt Glasser <glass...@wustl.edu>, "hcp-users@humanconnectome.org" <
>>> hcp-users@humanconnectome.org>
>>>
>>> Subject: Re: [HCP-Users] Workbench Tutorial Data
>>>
>>> Okay, I will use the regular data then and try to process it so that my
>>> algorithm runs on it. Do you guys know by any chance a good way to project
>>> the surface data contained in the cifti-files into a flat 2D map given in
>>> the flat gifti files? (I am trying to get the cifti data into a 2D matrix
>>> so I can convolve with a Gaussian kernel).
>>>
>>> On Tue, Jul 26, 2016 at 12:46 PM, Elam, Jennifer <e...@wustl.edu> wrote:
>>>
>>>> Hi Amrit,
>>>>
>>>> The data in Scene 4 of the WB Tutorial data is almost 5 years old now,
>>>> CP10101 is a subject from the pilot phase of the HCP, well before we were
>>>> releasing data and well before we were cleaning the data with FIX. The
>>>> reason that scene is there is simply to display the concept of BOLD
>>>> fluctuations in an fMRI time series-- it is not the really meant to be data
>>>> that you use any analysis tools on since we have since released several
>>>> improved versions of the fMRI data on now over 800 subjects.
>>>>
>>>>
>>>> The Group Average Workbench dataset available in ConnectomeDB
>>>> <https://db.humanconnectome.org/data/projects/HCP_900> is up to date
>>>> with data from the current 900 Subjects Release (of December 2015)
>>>>
>>>>
>>>> Yes, there is extensive documentation on the many individual subject
>>>> and group average analyzed data currently available for download in
>>>> ConnectomeDB in the 900 Subjects Reference Manual
>>>> <https://www.humanconnectome.org/documentation/S900/HCP_S900_Release_Reference_Manual.pdf>
>>>> (you can also get to it in the Documentation section of the HCP public
>>>> website) and more info in the publication references therein. It's a long
>>>> document, but there's a lot of valuable information there.
>>>>
>>>>
>>>> Best,
>>>>
>>>> Jenn
>>>>
>>>>
>>>> Jennifer Elam, Ph.D.
>>>> Scientific Outreach, Human Connectome Project
>>>> Washington University School of Medicine
>>>> Department of Neuroscience, Box 8108
>>>> 660 South Euclid Avenue
>>>> St. Louis, MO 63110
>>>> 314-362-9387
>>>> e...@wustl.edu
>>>> www.humanconnectome.org
>>>>
>>>> ------------------------------
>>>> *From:*hcp-users-boun...@humanconnectome.org <
>>>> hcp-users-boun...@humanconnectome.org> on behalf of Amrit Kashyap <
>>>> amrit...@gmail.com>
>>>> *Sent:* Tuesday, July 26, 2016 11:34:48 AM
>>>> *To:* Glasser, Matthew
>>>> *Cc:* hcp-users@humanconnectome.org
>>>> *Subject:* Re: [HCP-Users] Workbench Tutorial Data
>>>>
>>>> Yes, I am using that because I thought that was the most processed data
>>>> available. I think someone might have spatially smoothed this raw signal
>>>> and then added it to the tutorial data?
>>>>
>>>>
>>>> On Tue, Jul 26, 2016 at 11:08 AM, Glasser, Matthew <glass...@wustl.edu>
>>>> wrote:
>>>>
>>>>> Are you using the FIX cleaned HCP data? It will have _hp2000_clean in
>>>>> the file name.
>>>>>
>>>>> Peace,
>>>>>
>>>>> Matt.
>>>>>
>>>>> From: <hcp-users-boun...@humanconnectome.org> on behalf of Amrit
>>>>> Kashyap <amrit...@gmail.com>
>>>>> Date: Tuesday, July 26, 2016 at 9:52 AM
>>>>> To: "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org>
>>>>> Subject: [HCP-Users] Workbench Tutorial Data
>>>>>
>>>>> Hey HCP, this might not be the correct place to post this, but I have
>>>>> been using the rsfMRI data provided by the HCP workbench tutorial in scene
>>>>> 4 (CP10101). It looks like there has been some preprocessing done to the
>>>>> rsfMRI dtseries but I am not sure what it is. My own analysis techniques
>>>>> seem to work pretty well on this processed data but perform pretty poorly
>>>>> on the raw data. Is there any documentation/paper you could direct me to?
>>>>> Thanks
>>>>> Amrit
>>>>>
>>>>> _______________________________________________
>>>>> HCP-Users mailing list
>>>>> HCP-Users@humanconnectome.org
>>>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> The materials in this message are private and may contain Protected
>>>>> Healthcare Information or other information of a sensitive nature. If you
>>>>> are not the intended recipient, be advised that any unauthorized use,
>>>>> disclosure, copying or the taking of any action in reliance on the
>>>>> contents
>>>>> of this information is strictly prohibited. If you have received this
>>>>> email
>>>>> in error, please immediately notify the sender via telephone or return
>>>>> mail.
>>>>>
>>>>
>>>> _______________________________________________
>>>> HCP-Users mailing list
>>>> HCP-Users@humanconnectome.org
>>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users
>>>>
>>>
>>>
>>> ------------------------------
>>>
>>> The materials in this message are private and may contain Protected
>>> Healthcare Information or other information of a sensitive nature. If you
>>> are not the intended recipient, be advised that any unauthorized use,
>>> disclosure, copying or the taking of any action in reliance on the contents
>>> of this information is strictly prohibited. If you have received this email
>>> in error, please immediately notify the sender via telephone or return mail.
>>>
>>
>>
>> ------------------------------
>>
>> The materials in this message are private and may contain Protected
>> Healthcare Information or other information of a sensitive nature. If you
>> are not the intended recipient, be advised that any unauthorized use,
>> disclosure, copying or the taking of any action in reliance on the contents
>> of this information is strictly prohibited. If you have received this email
>> in error, please immediately notify the sender via telephone or return mail.
>>
>
>
> ------------------------------
>
> The materials in this message are private and may contain Protected
> Healthcare Information or other information of a sensitive nature. If you
> are not the intended recipient, be advised that any unauthorized use,
> disclosure, copying or the taking of any action in reliance on the contents
> of this information is strictly prohibited. If you have received this email
> in error, please immediately notify the sender via telephone or return mail.
>
_______________________________________________
HCP-Users mailing list
HCP-Users@humanconnectome.org
http://lists.humanconnectome.org/mailman/listinfo/hcp-users
