Hi Yaroslav, To reiterate something Mike Hodge mentioned, any given package only contains files for one subject. When interacting with ConnectomDB, the subjects to download, and the particular packages to download, are two separate and distinct choices.
cheers, -MH -- Michael Harms, Ph.D. ----------------------------------------------------------- Conte Center for the Neuroscience of Mental Disorders Washington University School of Medicine Department of Psychiatry, Box 8134 660 South Euclid Ave.Tel: 314-747-6173 St. Louis, MO 63110Email: mha...@wustl.edu On 12/6/16, 1:47 PM, "hcp-users-boun...@humanconnectome.org on behalf of Yaroslav Halchenko" <hcp-users-boun...@humanconnectome.org on behalf of y...@onerussian.com> wrote: On Tue, 06 Dec 2016, Elam, Jennifer wrote: > A listing of the by subject unpacked files available, organized by > modality and processing level, are available in Appendix 3 of the > Reference Manual. > The files are listed there as they unpack into a standard directory > structure. They are not organized by ConnectomeDB packages, per se, > because the listing is to be also applicable to users of Connectome >in a > Box and Amazon S3. If you really need a listing of the package >contents > themselves, we (Mike Hodge) can provide that separately. On Tue, 06 Dec 2016, Hodge, Michael wrote: > Yaroslav, > Separate packages are created for each subject. The list I sent just >listed packages for a couple of subjects to show you the files contained >in the packages by example. There aren't packages that correspond to the >unrelated groups. Each subject in the groups has a set of packages. I >could repeat the unzip search across all subjects if you wish, but it >would be a very large file. Dear Jennifer and Michael, Thank you for your replies! Let me may be describe my target use-case and why I was asking about packages, which may be would make situation a bit clearer. s3 HCP bucket provides convenient access to the dataset's individual files but they lack annotation on what package(s) (as shipped from db.) any particular file possibly belongs to. But such "packaging" is important meta-information since many folks analyze data from a particular "package". In datalad project we would like to provide access to data from HCP bucket, but also would like to allow users to specify "packages" -- as to which specific sub-datasets (e.g. not all subjects when not all subjects belong to a package) to install and which files to download. So it would look like following if we assume that 7T_MOVIE_2mm_preproc is a name of an example package which contains a subset of subjects with 7T movie "task" data. datalad search 7T_MOVIE_2mm_preproc | xargs datalad install to install those subjects' datasets (git-annex repositories without actual data by default), and then (hypothetical API) datalad get -r --annex-meta 7T_MOVIE_2mm_preproc to actually fetch data files present in the 7T_MOVIE_2mm_preproc package. Similarly, they could later run Since, I guess, you are composing those "packages" somehow already from a list of rules/files, I just thought that may be those could be shared, so we could embed that information in our annex HCP repositories and to not incur any additional "development/setup/maintenance cost" (as to dumping listing of generated .zip files). Then, if just plain .txt files with listings (unlike formatted pdfs -- easily machine readable), then people could also easily come up with their 1 line shell scripts to fetch corresponding to packages files from s3. So -- overall -- listings produced by Michael would work but I wondered if we could avoid (re)creating them and possibly make them even better for machine-parsing (e.g. one .txt file per each package which would include paths for files for all the subjects in that package). BTW -- 7T_MOVIE_2mm_preproc set of files is not yet on S3 bucket. When will that portion be uploaded? -- Yaroslav O. Halchenko Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
