Hi Mike, The only thing that is parallelized in the minimal preprocessing pipelines aside from specific FreeSurfer binaries is wb_command. wb_command defaults to using as many cores as the system has. More advanced pipelines that use matlab will adhere to matlab¹s parallelization behaviors.
Matt. On 4/27/17, 9:27 AM, "hcp-users-boun...@humanconnectome.org on behalf of Harms, Michael" <hcp-users-boun...@humanconnectome.org on behalf of mha...@wustl.edu> wrote: > >Matt, Tim, >What is parallelized in the fMRI pipelines? > >I would be good if we created a FAQ entry on the HCP Wiki detailing: >(1) Which aspects of the pipelines are parallelized and the default number >of cores used for each. >(2) Which wb_commands are parallelized and the default number of cores >used. >(3) How to explicitly control the number of cores used in both the >pipelines and wb_command. > >cheers, >-MH > >-- >Michael Harms, Ph.D. > >----------------------------------------------------------- >Conte Center for the Neuroscience of Mental Disorders >Washington University School of Medicine >Department of Psychiatry, Box 8134 >660 South Euclid Ave.Tel: 314-747-6173 >St. Louis, MO 63110Email: mha...@wustl.edu > > > > >On 4/27/17, 9:20 AM, "hcp-users-boun...@humanconnectome.org on behalf of >Gaurav Patel" <hcp-users-boun...@humanconnectome.org on behalf of >gauravpa...@gmail.com> wrote: > >For what its worth, on our workstations we run each subject's anatomy >pipeline as a separate thread, which takes 8-12 hours, then use the >parallelization to speed up the fMRI pipelines with either SGE/OGE or >ht_condor, with a thread devoted to each bold run being processed > >__________________________ > gaurav patel > gauravpa...@gmail.com > www.neurofreak.net > > > > >On Apr 27, 2017, at 9:44 AM, Harms, Michael wrote: > >> >> The number of windows you use to launch the processing is irrelevant. >>But if you only have 8 cores on your machine, and only want to use 7 of >>them, you can¹t run 50 subjects at once. >> >> In this case, where you have a bunch of subjects to process, I would run >>7 subjects at a time, while limiting each to just 1 core, by setting >>whatever variable in the pipeline that is used to limit/control the >>number of threads used for multi-thread capable binaries. >> >> cheers, >> -MH >> >> -- >> Michael Harms, Ph.D. >> ----------------------------------------------------------- >> Conte Center for the Neuroscience of Mental Disorders >> Washington University School of Medicine >> Department of Psychiatry, Box 8134 >> 660 South Euclid Ave. Tel: 314-747-6173 >> St. Louis, MO 63110 Email: mha...@wustl.edu >> >> From: <hcp-users-boun...@humanconnectome.org> on behalf of Vanessa K >><vanessakr1...@gmail.com> >> Date: Thursday, April 27, 2017 at 4:30 AM >> To: "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org> >> Subject: [HCP-Users] Fwd: Using multiple cores while running Structural >>Pipelines >> >> Alright, thanks! And just for clarification for dummies: if not >>installing something like grid_engine, will running all 50 subjects in >>one Terminal window or 10 subjects in each of five different windows be >>more efficient? Or doesn't it make any difference? And I estimate around >>20 days for 50 subjects (for the structural pipelines), is it a normal >>processing time? >> >> Thanks again! >> >> >> 2017-04-26 16:20 GMT+02:00 Glasser, Matthew <glass...@wustl.edu>: >>> I find it helps to install something like gridengine which interfaces >>>with fsl_sub. You can then queue up the jobs and they will all run >>>automatically. You can choose how many to run at a time. I believe you >>>don¹t need to specify an openmp flag unless you want fewer than all of >>>the cores. >>> >>> Peace, >>> >>> Matt. >>> >>> From: Vanessa K <vanessakr1...@gmail.com> >>> Date: Wednesday, April 26, 2017 at 4:39 AM >>> To: Matt Glasser <glass...@wustl.edu> >>> Cc: "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org> >>> Subject: Re: [HCP-Users] Using multiple cores while running Structural >>>Pipelines >>> >>> Thanks for your quick answer! >>> Just to make sure: it means I shouldn't specify -openmp 5 in the >>>recon-all line and that the multi-threading happens automatically? >>> So given that I have around 50 subjects who I need to run this on, >>>would the best way to accelerate the performance be to just open several >>>terminal windows (e.g. 5) and run ten subjects in each? Or would it make >>>no difference from running all 50 in one window? I considered looking >>>into GNU parallel, but not sure it would make sense. What would your >>>advice for such a number of subjects be? >>> >>> Thanks so much for your help. >>> >>> V >>> >>> 2017-04-25 16:55 GMT+02:00 Glasser, Matthew <glass...@wustl.edu>: >>>> PreFreeSurfer uses mostly single threaded tools. Some of FreeSurfer >>>>is multi-threaded (and importantly one binary has some issues with >>>>oversubscribed cores leading to much longer execution times). >>>>PostFreeSurfer is largely multi-threaded. In all cases, the pipelines >>>>use multiple threads by default if they can. >>>> >>>> Peace, >>>> >>>> Matt. >>>> >>>> From: <hcp-users-boun...@humanconnectome.org> on behalf of Vanessa K >>>><vanessakr1...@gmail.com> >>>> Date: Tuesday, April 25, 2017 at 7:36 AM >>>> To: "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org> >>>> Subject: [HCP-Users] Using multiple cores while running Structural >>>>Pipelines >>>> >>>> Dear users, >>>> >>>> it might be a silly question but I hope you can help. I have a >>>>computer with 8 cores and I would like to use 7 of them while running >>>>pipelines that take a lot of time (such as FreeSurfer). Is there any >>>>way to specify the core number when running the script? Or are there >>>>any tricks to speed up the processing time? >>>> >>>> Thanks a lot! >>>> >>>> V >>>> >>>> _______________________________________________ >>>> HCP-Users mailing list >>>> HCP-Users@humanconnectome.org >>>> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >>>> >>>> >>>> The materials in this message are private and may contain Protected >>>>Healthcare Information or other information of a sensitive nature. If >>>>you are not the intended recipient, be advised that any unauthorized >>>>use, disclosure, copying or the taking of any action in reliance on the >>>>contents of this information is strictly prohibited. If you have >>>>received this email in error, please immediately notify the sender via >>>>telephone or return mail. >>> >>> >>> >>> The materials in this message are private and may contain Protected >>>Healthcare Information or other information of a sensitive nature. If >>>you are not the intended recipient, be advised that any unauthorized >>>use, disclosure, copying or the taking of any action in reliance on the >>>contents of this information is strictly prohibited. If you have >>>received this email in error, please immediately notify the sender via >>>telephone or return mail. >> >> >> _______________________________________________ >> HCP-Users mailing list >> HCP-Users@humanconnectome.org >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >> >> >> The materials in this message are private and may contain Protected >>Healthcare Information or other information of a sensitive nature. If you >>are not the intended recipient, be advised that any unauthorized use, >>disclosure, copying or the taking of any action in reliance on the >>contents of this information is strictly prohibited. If you have received >>this email in error, please immediately notify the sender via telephone >>or return mail. >> _______________________________________________ >> HCP-Users mailing list >> HCP-Users@humanconnectome.org >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > > >_______________________________________________ >HCP-Users mailing list >HCP-Users@humanconnectome.org >http://lists.humanconnectome.org/mailman/listinfo/hcp-users > > >________________________________ >The materials in this message are private and may contain Protected >Healthcare Information or other information of a sensitive nature. If you >are not the intended recipient, be advised that any unauthorized use, >disclosure, copying or the taking of any action in reliance on the >contents of this information is strictly prohibited. If you have received >this email in error, please immediately notify the sender via telephone >or return mail. > >_______________________________________________ >HCP-Users mailing list >HCP-Users@humanconnectome.org >http://lists.humanconnectome.org/mailman/listinfo/hcp-users ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
