I went back and had a look at this data. It looks like an ICA+FIX like approach was applied, the mean grey matter time course was removed, and it was reconstructed from a d=50 ICA. I am assuming what you like about the dataset is the reduction to the 50 highest PCs, which removes the unstructured noise.
If you wish you can e-mail me off list and I can send you a dataset that has some of these properties but is processed using my current best approach for the purpose of testing your algorithm on it. As for your overall research goal, how is what you are doing meaningfully different than temporal ICA or PFMs? http://www.pnas.org/content/109/8/3131.short http://www.sciencedirect.com/science/article/pii/S1053811915000208 Peace, Matt. From: Amrit Kashyap <[email protected]<mailto:[email protected]>> Date: Tuesday, August 16, 2016 at 10:08 AM To: Matt Glasser <[email protected]<mailto:[email protected]>> Cc: "Elam, Jennifer" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [HCP-Users] Workbench Tutorial Data Hi Matthew, Thanks a lot for your response, I think I am understanding this a bit more. I am currently using an algorithm coming from fluid mechanics called Dynamic Mode Decomposition (http://www.sciencedirect.com/science/article/pii/S0165027015003829). The algorithm decomposes the dtseries into a sum of spatially oscillating modes. When I use the data from MSMAll_hp2000_clean.dtseries.nii, the algorithm is not able to decompose the signal. However, the pilot data (and only the pilot data, not the ica, normalized, or the raw) seems to decompose and I get spatial modes that are similar to functional connectivity networks seen in previous literature, as well as a distribution of frequencies that seem to be biological plausible. I think most of this is due to DMD being very sensitive, and if its off (which I am not quite sure how to define/characterize) the algorithm just gives me junk. Now, I am assuming (maybe incorrectly) that the middle paragraph in your last response, is what you used to process the pilot data. The mean centering and the normalization makes sense to me, but I am a bit confused on what "multiplying the bias field back into the data" refers to. I am confused what the bias field is (experimentally). Dividing by the variance definitely throws in a lot of noise (due to the fact that neighboring voxels probably should have similar std but since we have only limited data they are not). But I think, I would like to process the raw data ( MSMAll_hp2000_clean.dtseries.nii) with this method and see if my algorithm can decompose it. Thanks a lot for your help, Cheers Amrit On Mon, Aug 15, 2016 at 9:27 PM, Glasser, Matthew <[email protected]<mailto:[email protected]>> wrote: Well I processed the pilot data and have said that it is not recommended that you use that for analysis. The recommended starting data is the ICA+FIX cleaned data which has been registered with MSMAll and is named something like this: ${StudyFolder}/${Subject}/MNINonLinear/Results/${fMRIName}/${fMRIName}_Atlas_MSMAll_hp2000_clean.dtseries.nii If you are interested in looking at dense timeseries movies or concatenating across runs, you will want to remove the mean image from the data. Next you will need to decide whether you want to variance normalize the data (if you do, you will want to multiply the bias field back into the data and divide by the variance normalization map—these files are included in the same package as the above). Variance normalized data has an equal chance of false positives across space, but isn’t interpretable in terms of %BOLD deviation across space. The included bias field correction isn’t quite optimal and we haven’t had a chance to create the data that enables a better bias field correction. It is indeed true that if you reconstruct the data from the signal ICA components that a given run has, it will look a lot smoother spatially and temporally, because you will have thrown out all of the unstructured noise. This might be okay for individual subject analysis, but you wouldn’t want to do this for group analysis (because you would not be able to get the benefits of cross-subject averaging for bringing out weaker components). I’ve ended up doing something different which we call “Wishart Rolloff” which is described in this document: http://www.humanconnectome.org/documentation/mound-and-moat-effect.html This makes a bit fewer assumptions than does the ICA-based reconstruction approach, and so I have moved away from the ICA-recon approach. In any case, I don’t really understand exactly what you are trying to do with the data… Peace, Matt. From: Amrit Kashyap <[email protected]<mailto:[email protected]>> Date: Monday, August 15, 2016 at 10:35 AM To: Matt Glasser <[email protected]<mailto:[email protected]>> Cc: "Elam, Jennifer" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [HCP-Users] Workbench Tutorial Data [https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png] HCPfilterdNorm.mpg<https://drive.google.com/file/d/0B21oKdbljNLvaThnSy11anFqQzA/view?usp=drive_web>[X] [https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png] HCPicacleaned.mpg<https://drive.google.com/file/d/0B21oKdbljNLvOE9ISlVLdGt3cmc/view?usp=drive_web>[X] [https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png] HCPMSALLraw.mpg<https://drive.google.com/file/d/0B21oKdbljNLveUE1SzNHaFZrQzg/view?usp=drive_web>[X] [https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png] HCPpilot.mpg<https://drive.google.com/file/d/0B21oKdbljNLvYTlxR1dXbjlXY2M/view?usp=drive_web>[X] Hey HCP, I have downloaded the raw data that you have suggested and I tried many techniques unsuccessfully (filtering, smoothing, normalizing, ... etc) to try to transform the data into the signal that I have observed in the scene 4 data. I have included 4 files below to show the difference between the HCP pilot processing vs the raw data and vs my own processing. I believe that this signal (HCP pilot) is probably closer to the true signal that we are interested in, because it seems like the dynamics of the spatio-temporal pattern has normalized correctly over the entire brain, resulting in a smooth signal that transitions from one region to another. The raw signal (MSMAII in this case but ICA+fix is very similar) is not normalized across the brain space and shows most of the activity clustered around regions that inherently have more activity. Normalizing and filtering results in a signal that looks similar to the HCP pilot, but contains large amounts of spatial noise that is not seen in the HCP pilot data. I believe that probably the HCP pilot has been spatially filtered but I am not sure. I tried projecting the data into the 300 ICA components as well, and then reconstructing the signal from these components to clean the signal and it results in the fourth file which looks interesting but does not have the smooth dynamics seen from the pilot data. I think that someone has done an amazing job in post-processing the HCP pilot data and their work is a great interest for the scientific community. The last few weeks I have attempted to guess what has been done to the signal and try to apply it myself but this is not getting me very far. If you have any documentation about the pilot study or the code that generated the processing for the HCP pilot data I would be incredibly grateful since I am struggling to reproduce its results. For my present work, I am trying to extend the spatial functional connectivity maps into a temporal domain. For this I need very smooth signals both spatially as well as temporally and the HCP pilot data gives me promising results in decomposing the dynamics. Thanks so much for all the help you have already provided and appologize for the long email, Cheers Amrit On Wed, Jul 27, 2016 at 9:46 AM, Glasser, Matthew <[email protected]<mailto:[email protected]>> wrote: ICA+FIX for all usecases, but otherwise it depends on what you are doing. Peace, Matt. ________________________________ From: Amrit Kashyap <[email protected]<mailto:[email protected]>> Sent: Wednesday, July 27, 2016 8:20:51 AM To: Glasser, Matthew Cc: Elam, Jennifer; [email protected]<mailto:[email protected]> Subject: Re: [HCP-Users] Workbench Tutorial Data Awesome, thanks a bunch I will try it out. Just out of curiosity what preprocessing would you recommend to clean up the signal (filtering, interpolation, downsampling)? Cheers Amrit On Tue, Jul 26, 2016 at 1:50 PM, Glasser, Matthew <[email protected]<mailto:[email protected]>> wrote: There is a wb_command function that does this, wb_command -cifti-smoothing. We don’t recommend smoothing as a general preprocessing practice however. Peace, Matt. From: Amrit Kashyap <[email protected]<mailto:[email protected]>> Date: Tuesday, July 26, 2016 at 12:47 PM To: "Elam, Jennifer" <[email protected]<mailto:[email protected]>> Cc: Matt Glasser <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [HCP-Users] Workbench Tutorial Data Okay, I will use the regular data then and try to process it so that my algorithm runs on it. Do you guys know by any chance a good way to project the surface data contained in the cifti-files into a flat 2D map given in the flat gifti files? (I am trying to get the cifti data into a 2D matrix so I can convolve with a Gaussian kernel). On Tue, Jul 26, 2016 at 12:46 PM, Elam, Jennifer <[email protected]<mailto:[email protected]>> wrote: Hi Amrit, The data in Scene 4 of the WB Tutorial data is almost 5 years old now, CP10101 is a subject from the pilot phase of the HCP, well before we were releasing data and well before we were cleaning the data with FIX. The reason that scene is there is simply to display the concept of BOLD fluctuations in an fMRI time series-- it is not the really meant to be data that you use any analysis tools on since we have since released several improved versions of the fMRI data on now over 800 subjects. The Group Average Workbench dataset available in ConnectomeDB<https://db.humanconnectome.org/data/projects/HCP_900> is up to date with data from the current 900 Subjects Release (of December 2015) Yes, there is extensive documentation on the many individual subject and group average analyzed data currently available for download in ConnectomeDB in the 900 Subjects Reference Manual<https://www.humanconnectome.org/documentation/S900/HCP_S900_Release_Reference_Manual.pdf> (you can also get to it in the Documentation section of the HCP public website) and more info in the publication references therein. It's a long document, but there's a lot of valuable information there. Best, Jenn Jennifer Elam, Ph.D. Scientific Outreach, Human Connectome Project Washington University School of Medicine Department of Neuroscience, Box 8108 660 South Euclid Avenue St. Louis, MO 63110 314-362-9387<tel:314-362-9387> [email protected]<mailto:[email protected]> www.humanconnectome.org<http://www.humanconnectome.org/> ________________________________ From:[email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> on behalf of Amrit Kashyap <[email protected]<mailto:[email protected]>> Sent: Tuesday, July 26, 2016 11:34:48 AM To: Glasser, Matthew Cc: [email protected]<mailto:[email protected]> Subject: Re: [HCP-Users] Workbench Tutorial Data Yes, I am using that because I thought that was the most processed data available. I think someone might have spatially smoothed this raw signal and then added it to the tutorial data? On Tue, Jul 26, 2016 at 11:08 AM, Glasser, Matthew <[email protected]<mailto:[email protected]>> wrote: Are you using the FIX cleaned HCP data? It will have _hp2000_clean in the file name. Peace, Matt. From: <[email protected]<mailto:[email protected]>> on behalf of Amrit Kashyap <[email protected]<mailto:[email protected]>> Date: Tuesday, July 26, 2016 at 9:52 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: [HCP-Users] Workbench Tutorial Data Hey HCP, this might not be the correct place to post this, but I have been using the rsfMRI data provided by the HCP workbench tutorial in scene 4 (CP10101). It looks like there has been some preprocessing done to the rsfMRI dtseries but I am not sure what it is. My own analysis techniques seem to work pretty well on this processed data but perform pretty poorly on the raw data. Is there any documentation/paper you could direct me to? Thanks Amrit _______________________________________________ HCP-Users mailing list [email protected]<mailto:[email protected]> http://lists.humanconnectome.org/mailman/listinfo/hcp-users ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. _______________________________________________ HCP-Users mailing list [email protected]<mailto:[email protected]> http://lists.humanconnectome.org/mailman/listinfo/hcp-users ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. _______________________________________________ HCP-Users mailing list [email protected] http://lists.humanconnectome.org/mailman/listinfo/hcp-users
