Thanks for the insight, very interesting indeed. Elouan Keryell-Even
Software Engineer @ Atos Integration Toulouse, France 2015-12-09 11:08 GMT+01:00 Olivier Sallou <[email protected]>: > > > ------------------------------ > > *De: *"Elouan Keryell-Even" <[email protected]> > *À: *[email protected] > *Envoyé: *Mercredi 9 Décembre 2015 10:48:10 > *Objet: *Re: GenOuest makes use of Mesos > > Hi Olivier, > > I've just read the presentation on your project's webpage, and it seems > cool! I'm curious about the following mentioned feature: "Optional mount of > user home or other shared directories in container". Does your framework > take care of remotely copying the home directory onto the node where the > container is going to run, or does the directory have to be already > available (as a shared directory for example). > > We focus to "replace" classical computing cluster frameworks like Sun Grid > Engine. So home directories etc... are shared among nodes (nfs, etc...), > like in a usual cluster environment. > The mount of "user home dir" is based on an Auth/ACL plugin/configuration. > Admin specifies the available mounts for all users, or on a project basis. > Then the Auth/ACL plugin maps those volumes to real available directories > (if using the LDAP auth plugin, then get homeDirectory from LDAP and add as > a volume to container). > > Otherwise, we are currently using framework Chronos for our Batch oriented > containers, and your framework seems to fit the same spot. If you have some > experience with Chronos, I'd be interested in a brief comparison of both > frameworks. From what I understand, GODocker offers scheduling policy's > customization, which I don't think Chronos does. > > There is indeed scheduling algorithms plugin mechanism. An interesting one > is the fair-share policy imlpementation, where tasks are scheduler > (ordered) based on previous user/project consumption. > > Difference with Chronos, is Chronos is made for cron like jobs. GODocker > is a batch submission tool. User specifies the computing script he wants to > execute and it is executed "immediatly". The scheduling is a matter of > putting priorities when executing jobs (if users submits 1000 jobs but I > have only 100 slots available). We focus on users submitting "many" jobs or > many users submitting jobs, each job could be a computing task of a few > seconds or several days. > > > > Olivier > > > Thanks for your attention, > > Elouan Keryell-Even > > Software Engineer @ Atos Integration > > Toulouse, France > > 2015-12-08 14:49 GMT+01:00 Olivier Sallou <[email protected]>: > >> Hi, >> the GenOuest (http://www.genouest.org) academic lab is now using Mesos >> in production in its core facility to manage scientists computing tasks >> (for bioinformatics) >> To do so, we have developed a new mesos framework, GoDocker >> (http://www.genouest.org/godocker) to submit batch computing scripts on >> premises. It mounts users home directory or other shared resources to >> execute jobs in Docker containers, using Mesos as main scheduler. >> GoDocker schedules the jobs according to user/groups priorities and >> quotas and provides a CLI, a REST web interface and a partial DRMAA >> library support. Framewok is open source. >> >> Thanks for adding us to the Mesos fellows ;-) >> >> Regards >> >> Olivier (GenOuest core developer member) >> >> -- >> Olivier Sallou >> IRISA / University of Rennes 1 >> Campus de Beaulieu, 35000 RENNES - FRANCE >> Tel: 02.99.84.71.95 >> >> gpg key id: 4096R/326D8438 (keyring.debian.org) >> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 >> >> >> > >
