[galaxy-dev] Galaxy command line API information
Dear Sir or Madam, I am working for the University of Westminster on the ER-Flow project (http://www.erflow.eu/home). I'm currently investigating a way to use the galaxy workflow engine with a command line interface in order to support it within our project. The idea would be to develop a plugin that could make a link between our repository, where a galaxy workflow can be specified (workflow and inputs stored and output declared), and a galaxy server. We have a specific way our existing plugins work. It would be something like: ./executable -w workflow list of inputs And then, outputs are retrieved and returned (they don't need to be given as command line parameters). I had a quick look at the provided API within the galaxy project and I have some questions. I saw this is possible to import the workflow to the server, then to execute it. In my plugin, I would have to import the workflow, then run it with inputs listed in the executable command line, and after a successful execution, I would have to retrieve the outputs generated, then delete the created workflow and any files related that exist. Is it currently possible for me, in a short time, to develop such plugin performing operations previously described using the existing API? If yes, could you please provide me more information on the current API or some help on the different tools I should use? Many thanks and kind regards, Benoît Meilhac The University of Westminster is a charity and a company limited by guarantee. Registration number: 977818 England. Registered Office: 309 Regent Street, London W1B 2UW. This message and its attachments are private and confidential. If you have received this message in error, please notify the sender and remove it and its attachments from your system. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy command line API information
Dear Benoît Meilhac, In my plugin, I would have to import the workflow, then run it with inputs listed in the executable command line, and after a successful execution, I would have to retrieve the outputs generated, then delete the created workflow and any files related that exist. The inputs (datasets) on which the workflow will run have to exist within the Galaxy's system - you can't supply the files to the workflow directly. Is it currently possible for me, in a short time, to develop such plugin performing operations previously described using the existing API? If yes, could you please provide me more information on the current API or some help on the different tools I should use? You might consider the following flow. I believe all steps can be performed via API. 1. Import datasets into Galaxy ( using Tool API of the upload tool you have 2 options: either by linking the files - https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFilesor by direct uploading - the choice depends mostly on the size of inputs ) 2. Import workflow( https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.api.html#module-galaxy.webapps.galaxy.api.workflows ) 3. Run workflow on the datasets in Galaxy 4. Download results from history( https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.api.html#module-galaxy.webapps.galaxy.api.histories ) 5. Delete histories 6. Delete workflow Let me know how this works for you. thank you for using Galaxy! Martin On Fri, Feb 28, 2014 at 9:10 AM, Benoit Meilhac b.meil...@westminster.ac.uk wrote: Dear Sir or Madam, I am working for the University of Westminster on the ER-Flow project ( http://www.erflow.eu/home). I'm currently investigating a way to use the galaxy workflow engine with a command line interface in order to support it within our project. The idea would be to develop a plugin that could make a link between our repository, where a galaxy workflow can be specified (workflow and inputs stored and output declared), and a galaxy server. We have a specific way our existing plugins work. It would be something like: ./executable -w workflow list of inputs And then, outputs are retrieved and returned (they don't need to be given as command line parameters). I had a quick look at the provided API within the galaxy project and I have some questions. I saw this is possible to import the workflow to the server, then to execute it. In my plugin, I would have to import the workflow, then run it with inputs listed in the executable command line, and after a successful execution, I would have to retrieve the outputs generated, then delete the created workflow and any files related that exist. Is it currently possible for me, in a short time, to develop such plugin performing operations previously described using the existing API? If yes, could you please provide me more information on the current API or some help on the different tools I should use? Many thanks and kind regards, Benoît Meilhac The University of Westminster is a charity and a company limited by guarantee. Registration number: 977818 England. Registered Office: 309 Regent Street, London W1B 2UW. This message and its attachments are private and confidential. If you have received this message in error, please notify the sender and remove it and its attachments from your system. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy command line API information
Hello Benoît, I remember seeing projects trying to do something similar (store workflows internally and execute on Galaxy), like http://refinery-platform.org/, so it could be useful to take a look there too. I've been trying to do something similar but with the Java interface to the Galaxy API https://github.com/jmchilton/blend4j. I have no code to show off right now though :(. Aaron On Fri, Feb 28, 2014 at 11:36 AM, Martin Čech mar...@bx.psu.edu wrote: Dear Benoît Meilhac, In my plugin, I would have to import the workflow, then run it with inputs listed in the executable command line, and after a successful execution, I would have to retrieve the outputs generated, then delete the created workflow and any files related that exist. The inputs (datasets) on which the workflow will run have to exist within the Galaxy's system - you can't supply the files to the workflow directly. Is it currently possible for me, in a short time, to develop such plugin performing operations previously described using the existing API? If yes, could you please provide me more information on the current API or some help on the different tools I should use? You might consider the following flow. I believe all steps can be performed via API. 1. Import datasets into Galaxy ( using Tool API of the upload tool you have 2 options: either by linking the files - https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFilesor by direct uploading - the choice depends mostly on the size of inputs ) 2. Import workflow( https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.api.html#module-galaxy.webapps.galaxy.api.workflows ) 3. Run workflow on the datasets in Galaxy 4. Download results from history( https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.api.html#module-galaxy.webapps.galaxy.api.histories ) 5. Delete histories 6. Delete workflow Let me know how this works for you. thank you for using Galaxy! Martin On Fri, Feb 28, 2014 at 9:10 AM, Benoit Meilhac b.meil...@westminster.ac.uk wrote: Dear Sir or Madam, I am working for the University of Westminster on the ER-Flow project ( http://www.erflow.eu/home). I'm currently investigating a way to use the galaxy workflow engine with a command line interface in order to support it within our project. The idea would be to develop a plugin that could make a link between our repository, where a galaxy workflow can be specified (workflow and inputs stored and output declared), and a galaxy server. We have a specific way our existing plugins work. It would be something like: ./executable -w workflow list of inputs And then, outputs are retrieved and returned (they don't need to be given as command line parameters). I had a quick look at the provided API within the galaxy project and I have some questions. I saw this is possible to import the workflow to the server, then to execute it. In my plugin, I would have to import the workflow, then run it with inputs listed in the executable command line, and after a successful execution, I would have to retrieve the outputs generated, then delete the created workflow and any files related that exist. Is it currently possible for me, in a short time, to develop such plugin performing operations previously described using the existing API? If yes, could you please provide me more information on the current API or some help on the different tools I should use? Many thanks and kind regards, Benoît Meilhac The University of Westminster is a charity and a company limited by guarantee. Registration number: 977818 England. Registered Office: 309 Regent Street, London W1B 2UW. This message and its attachments are private and confidential. If you have received this message in error, please notify the sender and remove it and its attachments from your system. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use
Re: [galaxy-dev] galaxy command line
Sorry for the late response. Thanks for your help. On Wed, Jan 4, 2012 at 2:32 PM, Carlos Borroto carlos.borr...@gmail.comwrote: On Wed, Jan 4, 2012 at 2:25 PM, Ryan ngsbioinformat...@gmail.com wrote: On Wed, Jan 4, 2012 at 12:04 PM, Langhorst, Brad langho...@neb.com wrote: Usha: Galaxy is essentially a wrapper around other command line tools. So, while you could probably extract some of its pieces to run on the command line, I don't know why you would want to. I would argue it would be beneficial to invoke pipelines from the command line instead of being forced to use the web-based interface. command line is beneficial for large #'s of datasets that need to be analyzed. Ryan I think Ryan is right and I think that's exactly the niche for the API. Usha you could take a look into: http://wiki.g2.bx.psu.edu/Learn/API It seems there is very limited documentation, but it might help you to see if what you want is already possible. Regards, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy command line
Hello all, The documentation is on my todo list, and is fairly high up on it (and now it's a little higher). However, it is a wiki and everyone is encouraged to create an account and update the doc as you learn things. Dave C. On Wed, Jan 4, 2012 at 1:34 PM, Nate Coraor n...@bx.psu.edu wrote: On Jan 4, 2012, at 2:34 PM, Langhorst, Brad wrote: Ryan: I didn't understand Usha's question the same way you did. I agree that it's useful to run workflows in a more automated way. I have not run workflows using the Galaxy API yet, but I did see some documentation on it and plan to try it soon. http://wiki.g2.bx.psu.edu/Learn/API/Examples Maybe someone with more expertise has something more to say about this. Brad Hi All, It is indeed possible to run workflows from the command line via the API. Have a look at the sample script galaxy-dist/scripts/api/workflow_execute.py to see how it's done. Sorry for the lack of documentation, although this is finally in progress. --nate -- Brad Langhorst New England Biolabs langho...@neb.com From: Ryan ngsbioinformat...@gmail.com Date: Wed, 4 Jan 2012 14:25:44 -0500 To: Brad Langhorst langho...@neb.com Cc: Usha Reddy usha.reddy...@gmail.com, galaxy-...@bx.psu.edu galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] galaxy command line On Wed, Jan 4, 2012 at 12:04 PM, Langhorst, Brad langho...@neb.com wrote: Usha: Galaxy is essentially a wrapper around other command line tools. So, while you could probably extract some of its pieces to run on the command line, I don't know why you would want to. I would argue it would be beneficial to invoke pipelines from the command line instead of being forced to use the web-based interface. command line is beneficial for large #'s of datasets that need to be analyzed. Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy command line
Usha: Galaxy is essentially a wrapper around other command line tools. So, while you could probably extract some of its pieces to run on the command line, I don't know why you would want to. brad -- Brad Langhorst New England Biolabs langho...@neb.com From: Usha Reddy usha.reddy...@gmail.commailto:usha.reddy...@gmail.com Date: Tue, 3 Jan 2012 13:07:25 -0500 To: galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu Subject: [galaxy-dev] galaxy command line Galaxy is a web based platform. Can Galaxy be run as a command line tool? Thanks, Usha ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy command line
On Wed, Jan 4, 2012 at 12:04 PM, Langhorst, Brad langho...@neb.com wrote: Usha: Galaxy is essentially a wrapper around other command line tools. So, while you could probably extract some of its pieces to run on the command line, I don't know why you would want to. I would argue it would be beneficial to invoke pipelines from the command line instead of being forced to use the web-based interface. command line is beneficial for large #'s of datasets that need to be analyzed. Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy command line
On Wed, Jan 4, 2012 at 2:25 PM, Ryan ngsbioinformat...@gmail.com wrote: On Wed, Jan 4, 2012 at 12:04 PM, Langhorst, Brad langho...@neb.com wrote: Usha: Galaxy is essentially a wrapper around other command line tools. So, while you could probably extract some of its pieces to run on the command line, I don't know why you would want to. I would argue it would be beneficial to invoke pipelines from the command line instead of being forced to use the web-based interface. command line is beneficial for large #'s of datasets that need to be analyzed. Ryan I think Ryan is right and I think that's exactly the niche for the API. Usha you could take a look into: http://wiki.g2.bx.psu.edu/Learn/API It seems there is very limited documentation, but it might help you to see if what you want is already possible. Regards, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy command line
Ryan: I didn't understand Usha's question the same way you did. I agree that it's useful to run workflows in a more automated way. I have not run workflows using the Galaxy API yet, but I did see some documentation on it and plan to try it soon. http://wiki.g2.bx.psu.edu/Learn/API/Examples Maybe someone with more expertise has something more to say about this. Brad -- Brad Langhorst New England Biolabs langho...@neb.com From: Ryan ngsbioinformat...@gmail.commailto:ngsbioinformat...@gmail.com Date: Wed, 4 Jan 2012 14:25:44 -0500 To: Brad Langhorst langho...@neb.commailto:langho...@neb.com Cc: Usha Reddy usha.reddy...@gmail.commailto:usha.reddy...@gmail.com, galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] galaxy command line On Wed, Jan 4, 2012 at 12:04 PM, Langhorst, Brad langho...@neb.commailto:langho...@neb.com wrote: Usha: Galaxy is essentially a wrapper around other command line tools. So, while you could probably extract some of its pieces to run on the command line, I don't know why you would want to. I would argue it would be beneficial to invoke pipelines from the command line instead of being forced to use the web-based interface. command line is beneficial for large #'s of datasets that need to be analyzed. Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy command line
On Jan 4, 2012, at 2:34 PM, Langhorst, Brad wrote: Ryan: I didn't understand Usha's question the same way you did. I agree that it's useful to run workflows in a more automated way. I have not run workflows using the Galaxy API yet, but I did see some documentation on it and plan to try it soon. http://wiki.g2.bx.psu.edu/Learn/API/Examples Maybe someone with more expertise has something more to say about this. Brad Hi All, It is indeed possible to run workflows from the command line via the API. Have a look at the sample script galaxy-dist/scripts/api/workflow_execute.py to see how it's done. Sorry for the lack of documentation, although this is finally in progress. --nate -- Brad Langhorst New England Biolabs langho...@neb.com From: Ryan ngsbioinformat...@gmail.com Date: Wed, 4 Jan 2012 14:25:44 -0500 To: Brad Langhorst langho...@neb.com Cc: Usha Reddy usha.reddy...@gmail.com, galaxy-...@bx.psu.edu galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] galaxy command line On Wed, Jan 4, 2012 at 12:04 PM, Langhorst, Brad langho...@neb.com wrote: Usha: Galaxy is essentially a wrapper around other command line tools. So, while you could probably extract some of its pieces to run on the command line, I don't know why you would want to. I would argue it would be beneficial to invoke pipelines from the command line instead of being forced to use the web-based interface. command line is beneficial for large #'s of datasets that need to be analyzed. Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
