[galaxy-dev] Globus World 2014
Dear All I wanted to send a note to folks about Globus World 2014. I apologize before hand for spamming both the developer list and the users list also but I thought this may be relevant to folks on the lists. Please let me know if you have any questions. GlobusWorld is this year’s biggest gathering of all things Globus. GlobusWorld 2014 features a Using Globus Genomics to Accelerate Analysis Tutorial, and a full half day on Globus Genomicsin the main meeting, including a keynote by Nancy Cox and these accepted talks: Globus Genomics: Enabling high-throughput cloud-based analysis and management of NGS data for Translational Genomics research at Georgetown, by Yuriy Gusev, Improving next-generation sequencing variants identification in cancer genes using Globus Genomics, by Toshio Yoshimatsu Globus Genomics: A Medical Center's Bioinformatics Core Perspective, by Anoop Mayampurath Building a Low-budget Public Resource for Large-scale Proteomic Analyses, by Rama Raghavan Globus Genomics is a Globus and Galaxy based platform for genomic analysis. GlobusWorld is being held April 15-17, in Chicago. And, GCC2014 is a Silver Sponsor of GlobusWorld. -- Ravi K Madduri MCS, Argonne National Laboratory Computation Institute, University of Chicago ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Using Mesos to Enable distributed computing under Galaxy?
Kyle This is something I am very interested in. The three parts below make sense to me. I would be very happy to discuss further and provide any help to move this forward. Regards On Oct 26, 2013, at 2:43 PM, Kyle Ellrott wrote: > I think one of the aspects where Galaxy is a bit soft is the ability to do > distributed tasks. The current system of split/replicate/merge tasks based on > file type is a bit limited and hard for tool developers to expand upon. > Distributed computing is a non-trival thing to implement and I think it would > be a better use of our time to use an already existing framework. And it > would also mean one less API for tool writers to have to develop for. > I was wondering if anybody has looked at Mesos ( http://mesos.apache.org/ ). > You can see an overview of the Mesos architecture at > https://github.com/apache/mesos/blob/master/docs/Mesos-Architecture.md > The important thing about Mesos is that it provides an API for C/C++, > Java/Scala and Python to write distributed frameworks. There are already > implementations of frameworks for common parallel programming systems such as: > - Hadoop (https://github.com/mesos/hadoop) > - MPI > (https://github.com/apache/mesos/blob/master/docs/Running-torque-or-mpi-on-mesos.md) > - Spark (http://spark-project.org) > And you can find example Python framework at > https://github.com/apache/mesos/tree/master/src/examples/python > > Integration with Galaxy would have three parts: > 1) Add a system config variable to Galaxy called 'MESOS_URL' that is then > passed to tool wrappers and allows them to contact the local mesos > infrastructure (assuming the system has been configured) or pass a null if > the system isn't available. > 2) Write a tool runner that works as a mesos framework to executes single cpu > jobs on the distributed system. > 3) For instances where mesos is not available at a system wide level (say > they only have access to an SGE based cluster), but the user wants to run > distributed jobs, write a wrapper that can create a mesos cluster using the > existing queueing system. For example, right now I run a Mesos system under > the SGE queue system. > > I'm curious to see what other people think. > > Kyle > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ -- Ravi K Madduri MCS, Argonne National Laboratory Computation Institute, University of Chicago ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
