Re: [galaxy-dev] Unicode in tool stderr crashing galaxy? - solved, patch attached
At Thu, 10 Oct 2013 00:31:56 -0500, John Chilton wrote: > > Simon, using your test tools I was able to reproduce your errors. Pull > request 233 is my attempt to fix them with a lighter touch than just > immediately converting outputs to ascii. Do these patches work on your > end? > > Bjoern, do these patches fix your problem as well? > > If I don't hear anything from anyone, I will merge these changes in a few > days. Hi John, That works nicely, and is better than what I did, thanks! cheers, Simon ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unicode in tool stderr crashing galaxy? - solved, patch attached
Simon, using your test tools I was able to reproduce your errors. Pull
request 233 is my attempt to fix them with a lighter touch than just
immediately converting outputs to ascii. Do these patches work on your
end?
Bjoern, do these patches fix your problem as well?
If I don't hear anything from anyone, I will merge these changes in a few days.
Thanks guys,
-John
On Tue, Sep 24, 2013 at 8:29 PM, Guest, Simon
wrote:
> At Thu, 19 Sep 2013 11:44:35 +0200,
> Bjoern Gruening wrote:
>> Hi Simon,
>>
>> it's just a guess but can you try the attached patch if you have a
>> reproducible case. It is not tested, but at least it should point you in
>> the right direction. If its working, I will create a pull request.
>
> Hi Bjoern,
>
> Thanks for your reply and your patch. (Sorry for the delay in responding, I
> have been busy with other stuff.)
>
> While your patch didn't solve my problem, it certainly pointed me in the
> right direction.
>
> I made a new patch based on your idea, which I attach. The idea is, stdout
> and stderr of each tool run is converted from possible UTF-8 to ASCII, using
> a new function asciify which I added to Galaxy util.
>
> I made a repository on the Galaxy test toolshed called unicode_tester, to
> test this. This has 2 tools which generates some Unicode output, either on
> stdout or stderr accordingly. Log output shows that unicode characters are
> being removed.
>
> What else should I do to contribute this patch into Galaxy core?
>
> cheers,
> Simon
>
>>
>> > My optimism was short lived. I think I've hit a galaxy bug in Unicode
>> > processing.
>> > Having managed to get the deseq tool not to crash galaxy in the sqlalchemy
>> > layer,
>> > I now get a crash when clicking on the info->stderr for the tool output
>> > dataset, probably because it contains Unicode. Here's the traceback from
>> > paster.log:
>> >
>> >
>> > Exception happened during processing of request from ('127.0.0.1', 47124)
>> > Traceback (most recent call last):
>> > File
>> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
>> > line 1068, in process_request_in_thread
>> > self.finish_request(request, client_address)
>> > File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request
>> > self.RequestHandlerClass(request, client_address, self)
>> > File "/usr/lib64/python2.6/SocketServer.py", line 617, in __init__
>> > self.handle()
>> > File
>> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
>> > line 442, in handle
>> > BaseHTTPRequestHandler.handle(self)
>> > File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle
>> > self.handle_one_request()
>> > File
>> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
>> > line 437, in handle_one_request
>> > self.wsgi_execute()
>> > File
>> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
>> > line 290, in wsgi_execute
>> > self.wsgi_write_chunk(chunk)
>> > File
>> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
>> > line 150, in wsgi_write_chunk
>> > self.wfile.write(chunk)
>> > File "/usr/lib64/python2.6/socket.py", line 316, in write
>> > data = str(data) # XXX Should really reject non-string non-buffers
>> > UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
>> > position 169: ordinal not in range(128)
>> >
>> >
>> > I don't know where to look to find the actual stderr output. Any pointers?
>> >
>> > Is this actually a galaxy bug now?
>> >
>> > cheers,
>> > Simon
>
>
> ===
> Attention: The information contained in this message and/or attachments
> from AgResearch Limited is intended only for the persons or entities
> to which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of, or
> taking of any action in reliance upon, this information by persons or
> entities other than the intended recipients is prohibited by AgResearch
> Limited. If you have received this message in error, please notify the
> sender immediately.
> ===
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interfa
Re: [galaxy-dev] Unicode in tool stderr crashing galaxy? - solved, patch attached
At Thu, 19 Sep 2013 11:44:35 +0200,
Bjoern Gruening wrote:
> Hi Simon,
>
> it's just a guess but can you try the attached patch if you have a
> reproducible case. It is not tested, but at least it should point you in
> the right direction. If its working, I will create a pull request.
Hi Bjoern,
Thanks for your reply and your patch. (Sorry for the delay in responding, I
have been busy with other stuff.)
While your patch didn't solve my problem, it certainly pointed me in the right
direction.
I made a new patch based on your idea, which I attach. The idea is, stdout and
stderr of each tool run is converted from possible UTF-8 to ASCII, using a new
function asciify which I added to Galaxy util.
I made a repository on the Galaxy test toolshed called unicode_tester, to test
this. This has 2 tools which generates some Unicode output, either on stdout
or stderr accordingly. Log output shows that unicode characters are being
removed.
What else should I do to contribute this patch into Galaxy core?
cheers,
Simon
>
> > My optimism was short lived. I think I've hit a galaxy bug in Unicode
> > processing.
> > Having managed to get the deseq tool not to crash galaxy in the sqlalchemy
> > layer,
> > I now get a crash when clicking on the info->stderr for the tool output
> > dataset, probably because it contains Unicode. Here's the traceback from
> > paster.log:
> >
> >
> > Exception happened during processing of request from ('127.0.0.1', 47124)
> > Traceback (most recent call last):
> > File
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> > line 1068, in process_request_in_thread
> > self.finish_request(request, client_address)
> > File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request
> > self.RequestHandlerClass(request, client_address, self)
> > File "/usr/lib64/python2.6/SocketServer.py", line 617, in __init__
> > self.handle()
> > File
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> > line 442, in handle
> > BaseHTTPRequestHandler.handle(self)
> > File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle
> > self.handle_one_request()
> > File
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> > line 437, in handle_one_request
> > self.wsgi_execute()
> > File
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> > line 290, in wsgi_execute
> > self.wsgi_write_chunk(chunk)
> > File
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> > line 150, in wsgi_write_chunk
> > self.wfile.write(chunk)
> > File "/usr/lib64/python2.6/socket.py", line 316, in write
> > data = str(data) # XXX Should really reject non-string non-buffers
> > UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
> > position 169: ordinal not in range(128)
> >
> >
> > I don't know where to look to find the actual stderr output. Any pointers?
> >
> > Is this actually a galaxy bug now?
> >
> > cheers,
> > Simon
===
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
===
unicode-fix.patch
Description: Binary data
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
