At Thu, 19 Sep 2013 11:44:35 +0200,
Bjoern Gruening wrote:
> Hi Simon,
> 
> it's just a guess but can you try the attached patch if you have a
> reproducible case. It is not tested, but at least it should point you in
> the right direction. If its working, I will create a pull request.

Hi Bjoern,

Thanks for your reply and your patch.  (Sorry for the delay in responding, I 
have been busy with other stuff.)

While your patch didn't solve my problem, it certainly pointed me in the right 
direction.

I made a new patch based on your idea, which I attach.  The idea is, stdout and 
stderr of each tool run is converted from possible UTF-8 to ASCII, using a new 
function asciify which I added to Galaxy util.

I made a repository on the Galaxy test toolshed called unicode_tester, to test 
this.  This has 2 tools which generates some Unicode output, either on stdout 
or stderr accordingly.  Log output shows that unicode characters are being 
removed.

What else should I do to contribute this patch into Galaxy core?

cheers,
Simon

> 
> > My optimism was short lived.  I think I've hit a galaxy bug in Unicode 
> > processing.  
> > Having managed to get the deseq tool not to crash galaxy in the sqlalchemy 
> > layer, 
> > I now get a crash when clicking on the info->stderr for the tool output 
> > dataset, probably because it contains Unicode.  Here's the traceback from 
> > paster.log:
> > 
> > ----------------------------------------
> > Exception happened during processing of request from ('127.0.0.1', 47124)
> > Traceback (most recent call last):
> >   File 
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
> > line 1068, in process_request_in_thread
> >     self.finish_request(request, client_address)
> >   File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request
> >     self.RequestHandlerClass(request, client_address, self)
> >   File "/usr/lib64/python2.6/SocketServer.py", line 617, in __init__
> >     self.handle()
> >   File 
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
> > line 442, in handle
> >     BaseHTTPRequestHandler.handle(self)
> >   File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle
> >     self.handle_one_request()
> >   File 
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
> > line 437, in handle_one_request
> >     self.wsgi_execute()
> >   File 
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
> > line 290, in wsgi_execute
> >     self.wsgi_write_chunk(chunk)
> >   File 
> > "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
> > line 150, in wsgi_write_chunk
> >     self.wfile.write(chunk)
> >   File "/usr/lib64/python2.6/socket.py", line 316, in write
> >     data = str(data) # XXX Should really reject non-string non-buffers
> > UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in 
> > position 169: ordinal not in range(128)
> > ----------------------------------------
> > 
> > I don't know where to look to find the actual stderr output.  Any pointers?
> > 
> > Is this actually a galaxy bug now?
> > 
> > cheers,
> > Simon


=======================================================================
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
=======================================================================

Attachment: unicode-fix.patch
Description: Binary data

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to