We (Gabriel A. Devenyi and Jon Pipitone) would like to announce the 1.0 release <https://github.com/pipitone/qbatch/releases> of qbatch, a command-line tool for easily running a list of commands in parallel (serial farming) on a compute cluster. This tool takes the list of commands, divides them up into batches of arbitrary size, and then submits each batch as a separate job or as part of an array job. qbatch also gives you a consistent interface to use to submit commands on PBS and SGE clusters or locally, (support for others are planned/in testing, PRs welcome) while setting requirements for processors, walltime, memory and job dependencies. This tool can be used as a quick interface to spread work out on a cluster, or as the glue for for connecting a simple pipeline to a cluster (see https://github.com/CobraLab/antsRegistration-MAGeT for a sample implementation)
The target audience of qbatch is two-fold: it is immediately available for users of PBS or SGE clusters to simplify their job construction, in addition, through the use of environment variables, cluster administrations can craft a default qbatch deployment which allows new cluster users to quickly submit jobs which honours the cluster’s policies. For more information, check out our github webpage here: http://github.com/pipitone/qbatch qbatch is also available in pypi via pip install qbatch — Gabriel A. Devenyi B.Eng. Ph.D. Research Computing Associate Computational Brain Anatomy Laboratory Cerebral Imaging Center Douglas Mental Health University Institute Affiliate, Department of Psychiatry McGill University t: 514.761.6131x4781 e: [email protected]
_______________________________________________ Discuss mailing list [email protected] http://lists.software-carpentry.org/mailman/listinfo/discuss_lists.software-carpentry.org
