Hi Jack, I'm CC'ing the GC3Pie mailing-list -- feel free to continue the discussion there, as I guess this would be off-topic for the EasyBuild mailing-list.
On 12 August 2014 16:21, Jack Perdue <[email protected]> wrote: > Just curious... how does GC3Pie compare/contrast > to DRMAA? > > http://en.wikipedia.org/wiki/DRMAA They're quite different :-) 1) DRMAA is an API specification. Each batch system has its own implementation, mostly providing a C interface. Python or other language bindings are provided on top of it. GC3Pie is a Python framework, and you have to write your code in Python in order to use it. 2) DRMAA clients talk to a local batch system, i.e., DRMAA client code must run on a submit host. GC3Pie can use SSH or ARC transports and submit jobs to remote batch systems. Typically, users run a GC3Pie "master" script on their laptop and the scripts connect to cluster frontends over SSH to manage jobs. 3) A specific DRMAA library is tied to a single batch system implementation: you cannot have your DRMAA application talk to GridEngine and LSF at the same time. GC3Pie can submit to different "computational resources", each of which can use a different back-end. You can, for instance, run jobs on a SLURM cluster and on VMs on an OpenStack IaaS cloud. 4) Last time I checked, DRMAA had support for job arrays. GC3Pie supports more complex workflows (compose jobs in sequences and parallel sections), which can be altered by Python code when the workflow is running. 5) DRMAA allows running jobs asynchronously, via JobCallbacks GC3Pie has no notion of callbacks and must poll periodically for the job status. 6) DRMAA supports btach system features like advance reservation and machine information, which GC3Pie has no idea about. 7) GC3Pie has quite a larger memory footprint. You need approximately 1GB of RAM per 40'000 jobs. Disclaimer: I am a GC3Pie developer (hence biased) and I have no first-hand experience on DRMAA -- all of the above is based on my reading of the DRMAA specification document when we were deciding whether we should write a GC3Pie backend for DRMAA. I welcome any correction! Ciao, R
