Hi Jan-Mathijs, I've been pulling data from the Amazon S3 server (a valid option as explained here <https://wiki.humanconnectome.org/display/PublicData/Connecting+to+Connectome+Data+via+AWS>) because aspera connect works very poorly with ubuntu (i.e. nobody seems to be able to get the aspera downloader to work in linux, from this thread <https://www.mail-archive.com/[email protected]/msg02165.html>). I just assumed that the aspera downloads and the S3 server had all the same files.
The S3 server doesn't have anatomy packages. It has a folder for each subject with the various files in subfolders. Some subjects on the S3 server have T1w_acpc_dc_restore.nii.gz, and about a quarter of them don't. The s3 server doesn't have download packages like the aspera Connectome DB does (I don't see any, at any rate) Knowing this, I suppose I can just install aspera connect on a windows machine for the download and then transfer the files. So there is a work around. But from a documentation standpoint, the https://db.humanconnectome.org/ makes it seem like the Amazon S3 server and Aspera downloads are equivalent, when in actuality they don't provide access to all the same files. That's where all this confusion is coming from. >From a convenience standpoint, it would be nice if the S3 server had everything on it. The S3 server is a lot nicer (in my opinion) since it supports a command line interface. And because it works in linux without the user having to jump through hoops. The download speeds are pretty fantastic too. - Greg B. On Tue, Jan 12, 2016 at 12:32 PM, jan-mathijs schoffelen < [email protected]> wrote: > Hi Gregory, > > Although it is not clear to me why you would need the anatomical image > when processing the MEG data - the coregistration information is provided > in the MEG anatomy packages - the anatomicals you refer to > (T1w_acpc_dc_restore.nii.gz) are available from the subjects’ MEG_anatomy > packages. That is, I just downloaded MEG_anatomy for subject 105923 and it > seems to be there. Are you sure you have been looking in the right place? > > Best wishes, > > Jan-Mathijs > > > On Jan 12, 2016, at 3:59 PM, Gregory Butron <[email protected]> > wrote: > > Of course. The list below was generated by my python script, but I > manually checked a few of the subjects as well to confirm that it was > giving a true output. > > s3://hcp-openaccess/HCP/104012/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/105923/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/111514/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/146129/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/153732/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/156334/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/175540/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/192641/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/287248/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/512835/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/660951/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/662551/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/715950/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/725751/ does not have T1w folder > s3://hcp-openaccess/HCP/783462/ does not have T1w_acpc_dc_restore.nii.gz > s3://hcp-openaccess/HCP/825048/ does not have T1w_acpc_dc_restore.nii.gz > > This script only looked for the T1w_stuff_stuff.nii.gz file on subjects > that had resting state MEG data. I can check all the subjects if you like, > but below is a list of the subjects that the script checked for the > T1w_stuff.nii.gz file in. > > Searched subjets: > 100307/ > 102816/ > 104012/ > 105923/ > 106521/ > 108323/ > 109123/ > 111514/ > 113922/ > 116524/ > 125525/ > 133019/ > 140117/ > 146129/ > 149741/ > 151526/ > 153732/ > 156334/ > 158136/ > 162026/ > 166438/ > 172029/ > 175540/ > 177746/ > 181232/ > 182840/ > 185442/ > 187547/ > 189349/ > 191033/ > 191437/ > 191841/ > 192641/ > 195041/ > 200109/ > 204521/ > 205119/ > 212318/ > 221319/ > 233326/ > 250427/ > 255639/ > 287248/ > 293748/ > 352132/ > 352738/ > 433839/ > 500222/ > 512835/ > 559053/ > 568963/ > 599671/ > 601127/ > 660951/ > 662551/ > 665254/ > 680957/ > 715950/ > 725751/ > 783462/ > 814649/ > 825048/ > 872764/ > 877168/ > 898176/ > 912447/ > 917255/ > > > > On Tue, Jan 12, 2016 at 8:49 AM, Harms, Michael <[email protected]> wrote: > >> >> Every subject that goes through the structural pipeline will have a >> T1w_acpc_dc_restore.nii.gz file. Could you give some specific subject IDs >> for which that isn’t the case, so that our IT folks can look into what be >> going on with Amazon S3? >> >> cheers, >> -MH >> >> -- >> Michael Harms, Ph.D. >> ----------------------------------------------------------- >> Conte Center for the Neuroscience of Mental Disorders >> Washington University School of Medicine >> Department of Psychiatry, Box 8134 >> 660 South Euclid Ave. Tel: 314-747-6173 >> St. Louis, MO 63110 Email: [email protected] >> >> From: <[email protected]> on behalf of Gregory >> Butron <[email protected]> >> Date: Tuesday, January 12, 2016 at 7:57 AM >> To: "Glasser, Matthew" <[email protected]> >> Cc: "[email protected]" <[email protected]> >> Subject: Re: [HCP-Users] Missing T1w_acpc_dc_restore files >> >> Hello Matt, thanks for getting back to me. >> >> I have not downloaded all the structural files yet. I downloaded all the >> available files for a few subjects to test our software, but after testing >> I discovered that not all subjects have the same files available. Then I >> perused the files on the amazon S3 server to decide what I need to >> download. Grabbing the whole freesurfer directory for each subject (the >> directory containing the T1.mgz file) will probably work for me, but I'd >> like to know what sort of pre-processing has been done to it. >> >> If every subject had a T1w_acpc_dc_restore.nii.gz file I would be all >> set, because the reference manual explains exactly what that file is and >> what preprocessing and transforms have been applied to it. But since some >> of the subjects (~25%) don't have this file available, I need to use >> something else. >> >> When I say I can't find a file, or that a subject doesn't have a file, I >> mean that I don't see it on the amazon s3 server. >> >> Thanks, >> Greg B. >> >> On Mon, Jan 11, 2016 at 6:31 PM, Glasser, Matthew <[email protected]> >> wrote: >> >>> Did you also download the structural packages for those subjects? >>> >>> Peace, >>> >>> Matt. >>> >>> From: <[email protected]> on behalf of Gregory >>> Butron <[email protected]> >>> Date: Monday, January 11, 2016 at 1:54 PM >>> To: "[email protected]" <[email protected]> >>> Subject: [HCP-Users] Missing T1w_acpc_dc_restore files >>> >>> Hi All. >>> >>> I'm working with MEG data and I want to coregister it to the >>> T1w_acpc_dc_restore.nii.gz processed structure scan as recommended by the >>> reference manual. >>> >>> I downloaded the MEG data for the subjects who had resting state MEG >>> data, but I discovered that 16 of them do not have a >>> T1w_acpc_dc_restore.nii.gz file. >>> >>> Is there a reason why these subjects don't have this structure file? One >>> of the subjects (725751) don't have any MRI data at all, so I suppose that >>> is an excellent reason for not having the structure file, but I am a little >>> confused by the others. >>> >>> Since almost a quarter of the subjects with MEG data don't have the >>> T1w_acpc_dc_restore.nii.gz file, I would like to use an alternative. Every >>> subject that has a MRI scan has a T1.mgz file, so I think I can use this >>> file instead, but I have a few questions that I have not found answers to >>> in the documentation. >>> >>> First, what coordinate system does the T1.mgz file (i.e. >>> s3://hcp-openaccess/HCP/subjectID/T1w/subjectID/mri/T1.mgz) use? I >>> think I would relate it to the MEG data using the vox07mm transforms that >>> come with the MEG data, but I haven't found confirmation of that in the >>> documentation. >>> >>> Second, what sort of preprocessing has been done to the T1.mgz file? >>> Have the axes been realigned? Has it been mapped to fit any standard models? >>> >>> Thank you for your time, >>> Gregory Butron >>> >>> _______________________________________________ >>> HCP-Users mailing list >>> [email protected] >>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>> >>> >>> ------------------------------ >>> >>> The materials in this message are private and may contain Protected >>> Healthcare Information or other information of a sensitive nature. If you >>> are not the intended recipient, be advised that any unauthorized use, >>> disclosure, copying or the taking of any action in reliance on the contents >>> of this information is strictly prohibited. If you have received this email >>> in error, please immediately notify the sender via telephone or return mail. >>> >> >> _______________________________________________ >> HCP-Users mailing list >> [email protected] >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >> >> >> ------------------------------ >> >> The materials in this message are private and may contain Protected >> Healthcare Information or other information of a sensitive nature. If you >> are not the intended recipient, be advised that any unauthorized use, >> disclosure, copying or the taking of any action in reliance on the contents >> of this information is strictly prohibited. If you have received this email >> in error, please immediately notify the sender via telephone or return mail. >> > > _______________________________________________ > HCP-Users mailing list > [email protected] > http://lists.humanconnectome.org/mailman/listinfo/hcp-users > > > _______________________________________________ HCP-Users mailing list [email protected] http://lists.humanconnectome.org/mailman/listinfo/hcp-users
