Re: [galaxy-dev] Spaces in GenomeSpaces file URLs from genomespace_importer tool
Thanks for you help Peter.
I have reported the problem using the built in Galaxy reporting tool from
one of my failed jobs.
I also created a Trello bug submission.
Regards,
Marco
On Wed, Oct 21, 2015 at 10:15 AM, Peter Cock
wrote:
> Thanks Marco,
>
> OK, that seems to still be part of the Galaxy core:
>
>
> https://github.com/galaxyproject/galaxy/blob/dev/tools/genomespace/genomespace_importer.xml
>
> I think you need to file a bug with the main GitHub repository (or Trello).
>
> Looking at genomespace_importer.xml I don't see how the hidden URL
> parameter is used, but I would still guess it just needs a tag
> to to allow the percent sign for any URL-encoded characters to be passed.
>
> Peter
>
> On Wed, Oct 21, 2015 at 3:01 PM, Marco Ocana
> wrote:
> > Hi Peter,
> >
> > I am getting the metadata for the tool from:
> >
> > https://usegalaxy.org/api/tools/genomespace_importer
> >
> > Thanks
> >
> > Marco
> >
> > On Wed, Oct 21, 2015 at 8:01 AM, Peter Cock
> > wrote:
> >>
> >> Hi Marco,
> >>
> >> I'd lay money on the the tool parameter sanitization done in Cheetah
> >> intended to avoid any command insertion into the shell command.
> >> This can be configured within the tool definition using the
> >> tag set:
> >>
> >>
> >>
> https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E_tag_set
> >>
> >> Which genomespace_importer are you working from (URL please)?
> >>
> >> Peter
> >>
> >> On Wed, Oct 21, 2015 at 3:20 AM, Marco Ocana >
> >> wrote:
> >> > Hi,
> >> >
> >> > I am encountering a problem executing the genomespace_importer tool
> >> > using the Galaxy API.
> >> >
> >> > The tool works just fine to copy files in GenomeSpace into Galay a
> >> > history dataset EXCEPT when the GenomeSpace file URL includes a
> >> > space (" ") of any sort.
> >> >
> >> > Below is an example problem request (the GenomeSpace URL is public, so
> >> > you should be able to try it if you like).
> >> >
> >> > Note that the "URL" parameter includes a couple of %20 which are
> >> > supposed to url-encode the space character.
> >> >
> >> >
> >> > POST https://usegalaxy.org/api/tools?key=XX
> >> >
> >> > {
> >> >
> >> > "history_id": "039421d939e31170",
> >> >
> >> > "tool_id": "genomespace_importer",
> >> >
> >> > "inputs": {
> >> >
> >> > "URL":
> >> >
> >> > "
> https://dm.genomespace.org/datamanager/file/Home/Public/RecipeData/SequenceData_fa%20fasta%20fastq/RNA-Seq.fastq
> ",
> >> >
> >> > "gs-token": null
> >> >
> >> > }
> >> >
> >> > }
> >> >
> >> > The job is accepted by Galaxy but eventually it fails. An HTTP error
> >> > occurs during the execution of the job.
> >> >
> >> > I looked through the logs in GenomeSpace. Turns out that the URL being
> >> > used by Galaxy to do a GET on the file is
> >> >
> >> >
> >> >
> https://gsui.genomespace.org/datamanager/v1.0/file/Home/Public/RecipeData/SequenceData_faX20fastaX20fastq/RNA-Seq.fastq
> >> >
> >> > The "percents" are converted to X (I made the Xs bigger for emphasis).
> >> >
> >> > I attempted a couple of other experiments in the tool job submission,
> >> > using actual space characters instead of %20. I also tried using the +
> >> > instead of space.
> >> >
> >> > In both of these attempts, the exact same file path in the URL param
> was
> >> > used for the GenomeSpace GET, leading to a BAD REQUEST error with the
> >> > actual space character and NOT FOUND error with the + .
> >> >
> >> > So the question is: what kind of decoding is going on in the Galaxy
> >> > genomespace_importer tool and how do I get rid of the Xs?
> >> >
> >> > Thanks
> >> >
> >> > Marco
> >> >
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Spaces in GenomeSpaces file URLs from genomespace_importer tool
Hi Peter,
I am getting the metadata for the tool from:
https://usegalaxy.org/api/tools/genomespace_importer
Thanks
Marco
On Wed, Oct 21, 2015 at 8:01 AM, Peter Cock
wrote:
> Hi Marco,
>
> I'd lay money on the the tool parameter sanitization done in Cheetah
> intended to avoid any command insertion into the shell command.
> This can be configured within the tool definition using the
> tag set:
>
>
> https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E_tag_set
>
> Which genomespace_importer are you working from (URL please)?
>
> Peter
>
> On Wed, Oct 21, 2015 at 3:20 AM, Marco Ocana
> wrote:
> > Hi,
> >
> > I am encountering a problem executing the genomespace_importer tool using
> > the Galaxy API.
> >
> > The tool works just fine to copy files in GenomeSpace into Galay a
> history
> > dataset EXCEPT when the GenomeSpace file URL includes a space (" ") of
> any
> > sort.
> >
> > Below is an example problem request (the GenomeSpace URL is public, so
> you
> > should be able to try it if you like).
> >
> > Note that the "URL" parameter includes a couple of %20 which are
> supposed to
> > url-encode the space character.
> >
> >
> > POST https://usegalaxy.org/api/tools?key=XX
> >
> > {
> >
> > "history_id": "039421d939e31170",
> >
> > "tool_id": "genomespace_importer",
> >
> > "inputs": {
> >
> > "URL":
> > "
> https://dm.genomespace.org/datamanager/file/Home/Public/RecipeData/SequenceData_fa%20fasta%20fastq/RNA-Seq.fastq
> ",
> >
> > "gs-token": null
> >
> > }
> >
> > }
> >
> > The job is accepted by Galaxy but eventually it fails. An HTTP error
> occurs
> > during the execution of the job.
> >
> > I looked through the logs in GenomeSpace. Turns out that the URL being
> used
> > by Galaxy to do a GET on the file is
> >
> >
> https://gsui.genomespace.org/datamanager/v1.0/file/Home/Public/RecipeData/SequenceData_faX20fastaX20fastq/RNA-Seq.fastq
> >
> > The "percents" are converted to X (I made the Xs bigger for emphasis).
> >
> > I attempted a couple of other experiments in the tool job submission,
> using
> > actual space characters instead of %20. I also tried using the + instead
> of
> > space.
> >
> > In both of these attempts, the exact same file path in the URL param was
> > used for the GenomeSpace GET, leading to a BAD REQUEST error with the
> actual
> > space character and NOT FOUND error with the + .
> >
> > So the question is: what kind of decoding is going on in the Galaxy
> > genomespace_importer tool and how do I get rid of the Xs?
> >
> > Thanks
> >
> > Marco
> >
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client. To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > https://lists.galaxyproject.org/
> >
> > To search Galaxy mailing lists use the unified search at:
> > http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Spaces in GenomeSpaces file URLs from genomespace_importer tool
Hi,
I am encountering a problem executing the genomespace_importer tool using
the Galaxy API.
The tool works just fine to copy files in GenomeSpace into Galay a history
dataset EXCEPT when the GenomeSpace file URL includes a space (" ") of any
sort.
Below is an example problem request (the GenomeSpace URL is public, so you
should be able to try it if you like).
Note that the "URL" parameter includes a couple of %20 which are supposed
to url-encode the space character.
*POST https://usegalaxy.org/api/tools?key=XX
*
*{*
* "history_id": "039421d939e31170",*
* "tool_id": "genomespace_importer",*
* "inputs": {*
*"URL":
"https://dm.genomespace.org/datamanager/file/Home/Public/RecipeData/SequenceData_fa%20fasta%20fastq/RNA-Seq.fastq
",*
*"gs-token": null*
* }*
*}*
The job is accepted by Galaxy but eventually it fails. An HTTP error occurs
during the execution of the job.
I looked through the logs in GenomeSpace. Turns out that the URL being used
by Galaxy to do a GET on the file is
*https://gsui.genomespace.org/datamanager/v1.0/file/Home/Public/RecipeData/SequenceData_fa
X20fastaX20fastq/RNA-Seq.fastq*
The "percents" are converted to X (I made the Xs bigger for emphasis).
I attempted a couple of other experiments in the tool job submission, using
actual space characters instead of %20. I also tried using the + instead of
space.
In both of these attempts, the exact same file path in the URL param was
used for the GenomeSpace GET, leading to a BAD REQUEST error with the
actual space character and NOT FOUND error with the + .
So the question is: what kind of decoding is going on in the Galaxy
genomespace_importer tool and how do I get rid of the Xs?
Thanks
Marco
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] CORS requests to public Galaxy server
Hi, I am attempting to access the public usegalaxy.org server using the REST API from web browsers. The question: I am not seeing Access-Control-Allow-* headers in REST API responses. Are browser REST CORS requests supported by the Galaxy servers? More details: Here is an error reported by Chrome during a preflight OPTIONS request for a GET https://usegalaxy.org/api/tools?key=MYREALKEY request. XMLHttpRequest cannot load https://usegalaxy.org/api/tools?key=xx. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:63342' is therefore not allowed access. The response had HTTP status code 404. The request headers: 1. Accept: */* 2. Accept-Encoding: gzip, deflate, sdch 3. Accept-Language: en-US,en;q=0.8,es;q=0.6 4. Access-Control-Request-Headers: accept, content-type, gs-toolname 5. Access-Control-Request-Method: GET 6. Connection: keep-alive 7. Host: usegalaxy.org 8. Origin: http://localhost:63342 9. Referer: http://localhost:63342/GSRemoteComputatio/TestWidgetsWithRemoteJobs.html 10. User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36 The response headers from Galaxy: 1. Connection: keep-alive 2. Content-Encoding: gzip 3. Content-Type: text/html 4. Date: Tue, 13 Oct 2015 22:14:31 GMT 5. Server: nginx/1.4.7 6. Transfer-Encoding: chunked 7. Vary: Accept-Encoding Thanks Marco ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] API for job and workflow input metadata
Hi Eric, Thanks for the pointers. The info returned by these calls appear to cover my needs. Thanks again. Marco On Fri, Aug 21, 2015 at 1:59 PM, Eric Rasche wrote: > Hi Marco, > > You're looking for the tool build and show routes. > > Galaxy API: > > http://galaxy.readthedocs.org/en/master/api_doc.html#galaxy.webapps.galaxy.api.tools.ToolsController.build > > http://galaxy.readthedocs.org/en/master/api_doc.html#galaxy.webapps.galaxy.api.tools.ToolsController.show > > BioBlend: > > http://bioblend.readthedocs.org/en/latest/api_docs/galaxy/all.html#bioblend.galaxy.tools.ToolClient.show_tool > > Cheers, > Eric > > > > 2015-08-20 10:24 GMT-05:00 Marco Ocana : > >> Hi, >> >> I am working on writing some code that would allow for remotely executing >> jobs and workflows in Galaxy. >> Looking through the docs and examples that I have encountered, we are >> able to run specific tools if we know what are the inputs needed. >> Does the API (plain REST, bioblend, blend4j) allow for discovering what >> are the inputs needed to execute a specific tool? >> I'd be interested in things like argument name, type, label, description. >> And in the case where the input parameter is restricted to a number of >> options (e.g., pulldown selections), can I obtain those that would be >> available from the web pages? File types? etc. >> >> Big plus if we can find out similar information on the available output >> result. >> >> Thanks >> >> Marco >> >> >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> > > > > -- > Eric Rasche > Programmer II > > Center for Phage Technology > Rm 312A, BioBio > Texas A&M University > College Station, TX 77843 > 404-692-2048 > e...@tamu.edu > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] API for job and workflow input metadata
Hi, I am working on writing some code that would allow for remotely executing jobs and workflows in Galaxy. Looking through the docs and examples that I have encountered, we are able to run specific tools if we know what are the inputs needed. Does the API (plain REST, bioblend, blend4j) allow for discovering what are the inputs needed to execute a specific tool? I'd be interested in things like argument name, type, label, description. And in the case where the input parameter is restricted to a number of options (e.g., pulldown selections), can I obtain those that would be available from the web pages? File types? etc. Big plus if we can find out similar information on the available output result. Thanks Marco ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
