Thanks for the pointer, this is great stuff and I had not seen it before.
I’ve wanted to do something similar for a while now, and it’s becoming even
more important as we loop in GCE. I’ll definitely read through and play with
this and see what we might be able to use, contribute back to, and so on.
-Dannon
On Mon, Jan 26, 2015 at 12:28 PM, Brad Chapman chapm...@fastmail.com
wrote:
Enis, John and all;
I spotted ansible-cloudman on GitHub today which reminded me I've been
meaning to write about the approach we setup late last year to run bcbio
on AWS. It uses elasticluster (https://github.com/gc3-uzh-ch/elasticluster)
which has the advantage of being all Ansible scripts and bootstrapping
from standard images -- so no more making AMIs. It also uses SLURM
instead of SGE, which is a nice change.
We wrote an interface that automates all of the stuff you need to setup
on AWS: IAM users, VPCs and what not. It is a pretty streamlined process
from the command line, including specifying the cluster size and
stopping/starting it:
https://bcbio-nextgen.readthedocs.org/en/latest/contents/cloud.html#aws-setup
I also wrote up some benchmarking work using to give an idea of using
it in practice:
http://bcb.io/2014/12/19/awsbench/
All of the code is here:
https://github.com/chapmanb/bcbio-nextgen-vm
As always, happy to overlap/share with whatever y'all decide to
do. We could make bcbio specific stuff optional as needed, although it
is pretty lightweight -- just the driver scripts and a Docker image of
bcbio. It's basically a ready to use cluster with this little extra
added so hopefully could be useful for future plans with CloudMan.
Hope this is useful,
Brad
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
