Re: [HCP-Users] Missing T1w_acpc_dc_restore files

Tue, 12 Jan 2016 11:48:08 -0800


Hi Greg,
I believe the issue here is that some of the “MEG2” subjects that you’ve been looking at didn’t have their MRI structural packages released until the “S900” release, AND the Amazon S3 server is not yet synced with the S900 release.  We are currently working through some issues to get all the S900 data on Amazon.  When that is accomplished, I imagine that an announcement will go out to this user list.

So, yes, Amazon S3 will provide access to all the same files as the packages available from db.humanconnectome.org, but we aren’t quite there yet for data that was added as part of the S900 release.

cheers,
-MH

-- 
Michael Harms, Ph.D.
-----------------------------------------------------------
Conte Center for the Neuroscience of Mental Disorders
Washington University School of Medicine
Department of Psychiatry, Box 8134
660 South Euclid Ave. Tel: 314-747-6173
St. Louis, MO  63110 Email: [email protected]

From: Gregory Butron <[email protected]>
Date: Tuesday, January 12, 2016 at 1:15 PM
To: jan-mathijs schoffelen <[email protected]>
Cc: "Harms, Michael" <[email protected]>, "[email protected]" <[email protected]>
Subject: Re: [HCP-Users] Missing T1w_acpc_dc_restore files

Hi Jan-Mathijs,

I've been pulling data from the Amazon S3 server (a valid option as explained here) because aspera connect works very poorly with ubuntu (i.e. nobody seems to be able to get the aspera downloader to work in linux, from this thread). I just assumed that the aspera downloads and the S3 server had all the same files.

The S3 server doesn't have anatomy packages. It has a folder for each subject with the various files in subfolders. Some subjects on the S3 server have T1w_acpc_dc_restore.nii.gz, and about a quarter of them don't. The s3 server doesn't have download packages like the aspera Connectome DB does (I don't see any, at any rate)

Knowing this, I suppose I can just install aspera connect on a windows machine for the download and then transfer the files. So there is a work around.

But from a documentation standpoint, the https://db.humanconnectome.org/ makes it seem like the Amazon S3 server and Aspera downloads are equivalent, when in actuality they don't provide access to all the same files. That's where all this confusion is coming from.

From a convenience standpoint, it would be nice if the S3 server had everything on it. The S3 server is a lot nicer (in my opinion) since it supports a command line interface. And because it works in linux without the user having to jump through hoops. The download speeds are pretty fantastic too.

 - Greg B.

On Tue, Jan 12, 2016 at 12:32 PM, jan-mathijs schoffelen <[email protected]> wrote:
Hi Gregory,

Although it is not clear to me why you would need the anatomical image when processing the MEG data - the coregistration information is provided in the MEG anatomy packages - the anatomicals you refer to (T1w_acpc_dc_restore.nii.gz) are available from the subjects’ MEG_anatomy packages. That is, I just downloaded MEG_anatomy for subject 105923 and it seems to be there. Are you sure you have been looking in the right place?

Best wishes,

Jan-Mathijs
On Jan 12, 2016, at 3:59 PM, Gregory Butron <[email protected]> wrote:
Of course. The list below was generated by my python script, but I manually checked a few of the subjects as well to confirm that it was giving a true output.

s3://hcp-openaccess/HCP/104012/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/105923/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/111514/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/146129/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/153732/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/156334/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/175540/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/192641/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/287248/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/512835/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/660951/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/662551/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/715950/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/725751/ does not have T1w folder
s3://hcp-openaccess/HCP/783462/ does not have T1w_acpc_dc_restore.nii.gz
s3://hcp-openaccess/HCP/825048/ does not have T1w_acpc_dc_restore.nii.gz

This script only looked for the T1w_stuff_stuff.nii.gz file on subjects that had resting state MEG data. I can check all the subjects if you like, but below is a list of the subjects that the script checked for the T1w_stuff.nii.gz file in.

Searched subjets:
100307/
102816/
104012/
105923/
106521/
108323/
109123/
111514/
113922/
116524/
125525/
133019/
140117/
146129/
149741/
151526/
153732/
156334/
158136/
162026/
166438/
172029/
175540/
177746/
181232/
182840/
185442/
187547/
189349/
191033/
191437/
191841/
192641/
195041/
200109/
204521/
205119/
212318/
221319/
233326/
250427/
255639/
287248/
293748/
352132/
352738/
433839/
500222/
512835/
559053/
568963/
599671/
601127/
660951/
662551/
665254/
680957/
715950/
725751/
783462/
814649/
825048/
872764/
877168/
898176/
912447/
917255/



On Tue, Jan 12, 2016 at 8:49 AM, Harms, Michael <[email protected]> wrote:

Every subject that goes through the structural pipeline will have a T1w_acpc_dc_restore.nii.gz file.  Could you give some specific subject IDs for which that isn’t the case, so that our IT folks can look into what be going on with Amazon S3?

cheers,
-MH

-- 
Michael Harms, Ph.D.
-----------------------------------------------------------
Conte Center for the Neuroscience of Mental Disorders
Washington University School of Medicine
Department of Psychiatry, Box 8134
660 South Euclid Ave. Tel: 314-747-6173
St. Louis, MO  63110 Email: [email protected]

From: <[email protected]> on behalf of Gregory Butron <[email protected]>
Date: Tuesday, January 12, 2016 at 7:57 AM
To: "Glasser, Matthew" <[email protected]>
Cc: "[email protected]" <[email protected]>
Subject: Re: [HCP-Users] Missing T1w_acpc_dc_restore files

Hello Matt, thanks for getting back to me.

I have not downloaded all the structural files yet. I downloaded all the available files for a few subjects to test our software, but after testing I discovered that not all subjects have the same files available. Then I perused the files on the amazon S3 server to decide what I need to download. Grabbing the whole freesurfer directory for each subject (the directory containing the T1.mgz file) will probably work for me, but I'd like to know what sort of pre-processing has been done to it.

If every subject had a T1w_acpc_dc_restore.nii.gz file I would be all set, because the reference manual explains exactly what that file is and what preprocessing and transforms have been applied to it. But since some of the subjects (~25%) don't have this file available, I need to use something else.

When I say I can't find a file, or that a subject doesn't have a file, I mean that I don't see it on the amazon s3 server.

Thanks,
Greg B.

On Mon, Jan 11, 2016 at 6:31 PM, Glasser, Matthew <[email protected]> wrote:
Did you also download the structural packages for those subjects?

Peace,

Matt.

From: <[email protected]> on behalf of Gregory Butron <[email protected]>
Date: Monday, January 11, 2016 at 1:54 PM
To: "[email protected]" <[email protected]>
Subject: [HCP-Users] Missing T1w_acpc_dc_restore files

Hi All.

I'm working with MEG data and I want to coregister it to the T1w_acpc_dc_restore.nii.gz processed structure scan as recommended by the reference manual.

I downloaded the MEG data for the subjects who had resting state MEG data, but I discovered that 16 of them do not have a T1w_acpc_dc_restore.nii.gz file.

Is there a reason why these subjects don't have this structure file? One of the subjects (725751) don't have any MRI data at all, so I suppose that is an excellent reason for not having the structure file, but I am a little confused by the others. 

Since almost a quarter of the subjects with MEG data don't have the T1w_acpc_dc_restore.nii.gz file, I would like to use an alternative. Every subject that has a MRI scan has a T1.mgz file, so I think I can use this file instead, but I have a few questions that I have not found answers to in the documentation.

First, what coordinate system does the T1.mgz file (i.e. s3://hcp-openaccess/HCP/subjectID/T1w/subjectID/mri/T1.mgz) use? I think I would relate it to the MEG data using the vox07mm transforms that come with the MEG data, but I haven't found confirmation of that in the documentation.

Second, what sort of preprocessing has been done to the T1.mgz file? Have the axes been realigned? Has it been mapped to fit any standard models?

Thank you for your time,
Gregory Butron

_______________________________________________
HCP-Users mailing list
[email protected]
http://lists.humanconnectome.org/mailman/listinfo/hcp-users

 

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.


_______________________________________________
HCP-Users mailing list
[email protected]
http://lists.humanconnectome.org/mailman/listinfo/hcp-users

 

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.


_______________________________________________
HCP-Users mailing list
[email protected]
http://lists.humanconnectome.org/mailman/listinfo/hcp-users

 

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

_______________________________________________
HCP-Users mailing list
[email protected]
http://lists.humanconnectome.org/mailman/listinfo/hcp-users

Reply via email to