Hi Juan Gabriel, You have many choices available to you when it comes to running tasks either using an on-prem cluster or cloud-based resources.
As Riccardo points out, we have documented the "how-to" get started using Elasticluster and Grid Engine because many existing bioinformatics scripts have been written for Grid Engine and many people have experience with it at their own institution. For development of new pipelines for a cloud environment such as Google Cloud, you have newer choices, such as: * The Broad Institute's workflow runner, Cromwell <https://github.com/broadinstitute/cromwell>, which supports the Google Pipelines API <https://cloud.google.com/genomics/v1alpha2/pipelines> as well as cluster managers like Grid Engine * dsub <https://github.com/googlegenomics/dsub>, a simple command-line tool that today supports running tasks using the Google Pipelines API and is intended to support other providers in the future Hope that helps, -Matt On Wednesday, June 21, 2017 at 3:31:25 PM UTC-7, Juan Gabriel Giarrizzo wrote: > > Hello, > > I used elasticluster to create a grid engine cluster in GCP following this > instructions > http://googlegenomics.readthedocs.io/en/latest/use_cases/setup_gridengine_cluster_on_compute_engine/ > > Now I have a working cluster but have no idea how to send my jobs and > distribute the load across the nodes!! > > I have been trying using qsub commands and shell scripts to run my > analysis tools but I still see that only one machine gets used and that is > maxing out the CPU. > > My questions are > > 1. What is a good and well documented scheduler to submit my jobs? > 2. How do I manage to distribute the job across nodes, and actually > benefit from having a cluster? > > Thank you for help > -- You received this message because you are subscribed to the Google Groups "elasticluster" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
