Hooray, I think that resolves all the confusion then. Thank you for the clarification.
On Tue, Jan 12, 2016 at 1:46 PM, Harms, Michael <[email protected]> wrote: > > Hi Greg, > I believe the issue here is that some of the “MEG2” subjects that you’ve > been looking at didn’t have their MRI structural packages released until > the “S900” release, AND the Amazon S3 server is not yet synced with the > S900 release. We are currently working through some issues to get all the > S900 data on Amazon. When that is accomplished, I imagine that an > announcement will go out to this user list. > > So, yes, Amazon S3 will provide access to all the same files as the > packages available from db.humanconnectome.org, but we aren’t quite there > yet for data that was added as part of the S900 release. > > cheers, > -MH > > -- > Michael Harms, Ph.D. > ----------------------------------------------------------- > Conte Center for the Neuroscience of Mental Disorders > Washington University School of Medicine > Department of Psychiatry, Box 8134 > 660 South Euclid Ave. Tel: 314-747-6173 > St. Louis, MO 63110 Email: [email protected] > > From: Gregory Butron <[email protected]> > Date: Tuesday, January 12, 2016 at 1:15 PM > To: jan-mathijs schoffelen <[email protected]> > Cc: "Harms, Michael" <[email protected]>, "[email protected]" < > [email protected]> > > Subject: Re: [HCP-Users] Missing T1w_acpc_dc_restore files > > Hi Jan-Mathijs, > > I've been pulling data from the Amazon S3 server (a valid option as > explained here > <https://wiki.humanconnectome.org/display/PublicData/Connecting+to+Connectome+Data+via+AWS>) > because aspera connect works very poorly with ubuntu (i.e. nobody seems to > be able to get the aspera downloader to work in linux, from this thread > <https://www.mail-archive.com/[email protected]/msg02165.html>). > I just assumed that the aspera downloads and the S3 server had all the same > files. > > The S3 server doesn't have anatomy packages. It has a folder for each > subject with the various files in subfolders. Some subjects on the S3 > server have T1w_acpc_dc_restore.nii.gz, and about a quarter of them don't. > The s3 server doesn't have download packages like the aspera Connectome DB > does (I don't see any, at any rate) > > Knowing this, I suppose I can just install aspera connect on a windows > machine for the download and then transfer the files. So there is a work > around. > > But from a documentation standpoint, the https://db.humanconnectome.org/ > makes it seem like the Amazon S3 server and Aspera downloads are > equivalent, when in actuality they don't provide access to all the same > files. That's where all this confusion is coming from. > > From a convenience standpoint, it would be nice if the S3 server had > everything on it. The S3 server is a lot nicer (in my opinion) since it > supports a command line interface. And because it works in linux without > the user having to jump through hoops. The download speeds are pretty > fantastic too. > > - Greg B. > > On Tue, Jan 12, 2016 at 12:32 PM, jan-mathijs schoffelen < > [email protected]> wrote: > >> Hi Gregory, >> >> Although it is not clear to me why you would need the anatomical image >> when processing the MEG data - the coregistration information is provided >> in the MEG anatomy packages - the anatomicals you refer to >> (T1w_acpc_dc_restore.nii.gz) are available from the subjects’ MEG_anatomy >> packages. That is, I just downloaded MEG_anatomy for subject 105923 and it >> seems to be there. Are you sure you have been looking in the right place? >> >> Best wishes, >> >> Jan-Mathijs >> >> >> On Jan 12, 2016, at 3:59 PM, Gregory Butron <[email protected]> >> wrote: >> >> Of course. The list below was generated by my python script, but I >> manually checked a few of the subjects as well to confirm that it was >> giving a true output. >> >> s3://hcp-openaccess/HCP/104012/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/105923/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/111514/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/146129/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/153732/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/156334/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/175540/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/192641/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/287248/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/512835/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/660951/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/662551/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/715950/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/725751/ does not have T1w folder >> s3://hcp-openaccess/HCP/783462/ does not have T1w_acpc_dc_restore.nii.gz >> s3://hcp-openaccess/HCP/825048/ does not have T1w_acpc_dc_restore.nii.gz >> >> This script only looked for the T1w_stuff_stuff.nii.gz file on subjects >> that had resting state MEG data. I can check all the subjects if you like, >> but below is a list of the subjects that the script checked for the >> T1w_stuff.nii.gz file in. >> >> Searched subjets: >> 100307/ >> 102816/ >> 104012/ >> 105923/ >> 106521/ >> 108323/ >> 109123/ >> 111514/ >> 113922/ >> 116524/ >> 125525/ >> 133019/ >> 140117/ >> 146129/ >> 149741/ >> 151526/ >> 153732/ >> 156334/ >> 158136/ >> 162026/ >> 166438/ >> 172029/ >> 175540/ >> 177746/ >> 181232/ >> 182840/ >> 185442/ >> 187547/ >> 189349/ >> 191033/ >> 191437/ >> 191841/ >> 192641/ >> 195041/ >> 200109/ >> 204521/ >> 205119/ >> 212318/ >> 221319/ >> 233326/ >> 250427/ >> 255639/ >> 287248/ >> 293748/ >> 352132/ >> 352738/ >> 433839/ >> 500222/ >> 512835/ >> 559053/ >> 568963/ >> 599671/ >> 601127/ >> 660951/ >> 662551/ >> 665254/ >> 680957/ >> 715950/ >> 725751/ >> 783462/ >> 814649/ >> 825048/ >> 872764/ >> 877168/ >> 898176/ >> 912447/ >> 917255/ >> >> >> >> On Tue, Jan 12, 2016 at 8:49 AM, Harms, Michael <[email protected]> wrote: >> >>> >>> Every subject that goes through the structural pipeline will have a >>> T1w_acpc_dc_restore.nii.gz file. Could you give some specific subject IDs >>> for which that isn’t the case, so that our IT folks can look into what be >>> going on with Amazon S3? >>> >>> cheers, >>> -MH >>> >>> -- >>> Michael Harms, Ph.D. >>> ----------------------------------------------------------- >>> Conte Center for the Neuroscience of Mental Disorders >>> Washington University School of Medicine >>> Department of Psychiatry, Box 8134 >>> 660 South Euclid Ave. Tel: 314-747-6173 >>> St. Louis, MO 63110 Email: [email protected] >>> >>> From: <[email protected]> on behalf of Gregory >>> Butron <[email protected]> >>> Date: Tuesday, January 12, 2016 at 7:57 AM >>> To: "Glasser, Matthew" <[email protected]> >>> Cc: "[email protected]" <[email protected]> >>> Subject: Re: [HCP-Users] Missing T1w_acpc_dc_restore files >>> >>> Hello Matt, thanks for getting back to me. >>> >>> I have not downloaded all the structural files yet. I downloaded all the >>> available files for a few subjects to test our software, but after testing >>> I discovered that not all subjects have the same files available. Then I >>> perused the files on the amazon S3 server to decide what I need to >>> download. Grabbing the whole freesurfer directory for each subject (the >>> directory containing the T1.mgz file) will probably work for me, but I'd >>> like to know what sort of pre-processing has been done to it. >>> >>> If every subject had a T1w_acpc_dc_restore.nii.gz file I would be all >>> set, because the reference manual explains exactly what that file is and >>> what preprocessing and transforms have been applied to it. But since some >>> of the subjects (~25%) don't have this file available, I need to use >>> something else. >>> >>> When I say I can't find a file, or that a subject doesn't have a file, I >>> mean that I don't see it on the amazon s3 server. >>> >>> Thanks, >>> Greg B. >>> >>> On Mon, Jan 11, 2016 at 6:31 PM, Glasser, Matthew <[email protected]> >>> wrote: >>> >>>> Did you also download the structural packages for those subjects? >>>> >>>> Peace, >>>> >>>> Matt. >>>> >>>> From: <[email protected]> on behalf of Gregory >>>> Butron <[email protected]> >>>> Date: Monday, January 11, 2016 at 1:54 PM >>>> To: "[email protected]" <[email protected]> >>>> Subject: [HCP-Users] Missing T1w_acpc_dc_restore files >>>> >>>> Hi All. >>>> >>>> I'm working with MEG data and I want to coregister it to the >>>> T1w_acpc_dc_restore.nii.gz processed structure scan as recommended by the >>>> reference manual. >>>> >>>> I downloaded the MEG data for the subjects who had resting state MEG >>>> data, but I discovered that 16 of them do not have a >>>> T1w_acpc_dc_restore.nii.gz file. >>>> >>>> Is there a reason why these subjects don't have this structure file? >>>> One of the subjects (725751) don't have any MRI data at all, so I suppose >>>> that is an excellent reason for not having the structure file, but I am a >>>> little confused by the others. >>>> >>>> Since almost a quarter of the subjects with MEG data don't have the >>>> T1w_acpc_dc_restore.nii.gz file, I would like to use an alternative. Every >>>> subject that has a MRI scan has a T1.mgz file, so I think I can use this >>>> file instead, but I have a few questions that I have not found answers to >>>> in the documentation. >>>> >>>> First, what coordinate system does the T1.mgz file (i.e. >>>> s3://hcp-openaccess/HCP/subjectID/T1w/subjectID/mri/T1.mgz) use? I >>>> think I would relate it to the MEG data using the vox07mm transforms that >>>> come with the MEG data, but I haven't found confirmation of that in the >>>> documentation. >>>> >>>> Second, what sort of preprocessing has been done to the T1.mgz file? >>>> Have the axes been realigned? Has it been mapped to fit any standard >>>> models? >>>> >>>> Thank you for your time, >>>> Gregory Butron >>>> >>>> _______________________________________________ >>>> HCP-Users mailing list >>>> [email protected] >>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>>> >>>> >>>> ------------------------------ >>>> >>>> The materials in this message are private and may contain Protected >>>> Healthcare Information or other information of a sensitive nature. If you >>>> are not the intended recipient, be advised that any unauthorized use, >>>> disclosure, copying or the taking of any action in reliance on the contents >>>> of this information is strictly prohibited. If you have received this email >>>> in error, please immediately notify the sender via telephone or return >>>> mail. >>>> >>> >>> _______________________________________________ >>> HCP-Users mailing list >>> [email protected] >>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>> >>> >>> ------------------------------ >>> >>> The materials in this message are private and may contain Protected >>> Healthcare Information or other information of a sensitive nature. If you >>> are not the intended recipient, be advised that any unauthorized use, >>> disclosure, copying or the taking of any action in reliance on the contents >>> of this information is strictly prohibited. If you have received this email >>> in error, please immediately notify the sender via telephone or return mail. >>> >> >> _______________________________________________ >> HCP-Users mailing list >> [email protected] >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >> >> >> > > ------------------------------ > > The materials in this message are private and may contain Protected > Healthcare Information or other information of a sensitive nature. If you > are not the intended recipient, be advised that any unauthorized use, > disclosure, copying or the taking of any action in reliance on the contents > of this information is strictly prohibited. If you have received this email > in error, please immediately notify the sender via telephone or return mail. > _______________________________________________ HCP-Users mailing list [email protected] http://lists.humanconnectome.org/mailman/listinfo/hcp-users
