Re: [galaxy-dev] error with unicode output

2013-12-06 Thread Christian Hundsrucker
I tried different combinations ... The DEFAULT_ENCODING = 'latin-1' in lib/galaxy/util/__init__.py is what worked! some background information: Yes, we also use a MySQL database. All encodings (server, R/Bioconductor, used editors, ... ) are UTF-8. The changeset

Re: [galaxy-dev] error with unicode output

2013-12-06 Thread David Hoover
Adding ?charset=utf8 to the connection string worked! However, there is one really weird side effect that probably has nothing to do with utf8. The tool creates two pdf files and one zip file as output. For the two pdf files, the expansion of the dataset in the history bar shows 'Image in

Re: [galaxy-dev] error with unicode output

2013-12-06 Thread John Chilton
Okay all, Christian followed up off list and mentioned that his columns had not been UTF-8 encoded. So I think the best thing to do is use Postgres, but if you have to use MySQL make sure columns are UTF-8 encoded with the newest Galaxy and add ?charset=utf8 to your database connection string. I

[galaxy-dev] error with unicode output

2013-12-05 Thread David Hoover
I have installed the ngsplot galaxy tool from http://code.google.com/p/ngsplot. This tool creates a set of three pdf files. In older versions of Galaxy, the tool ran correctly with no problems. A recent update broke the tool. The job runs but is unable to finish. Here is the error

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread Christian Hundsrucker
Hi David, hi all! I have a similar/the same issue in another setting... galaxy/galaxy_dist/lib/galaxy/jobs/runners/local.py, line 116, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) [...]

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread John Chilton
David, Christian, Very sorry about this - this is probably related to fixing some other errors - http://dev.list.galaxyproject.org/Unicode-in-tool-stderr-crashing-galaxy-tt4661749.html#a4661750. I will try to look into this. Christian - what database are targeting? Is it MySQL as well? David -

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread John Chilton
Actually, can you verify that this commit https://bitbucket.org/galaxy/galaxy-central/commits/e786022dc67ed918050bd81b9ac679ac958e4f75 is in your distribution and if it is try changing: DEFAULT_ENCODING = 'utf-8' in lib/galaxy/util.py to DEFAULT_ENCODING = 'latin-1' If that works then - I can

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread David Hoover
My version is 11216:c458a0fe1ba8. Does this cover the commit? On Dec 5, 2013, at 12:32 PM, John Chilton wrote: Actually, can you verify that this commit https://bitbucket.org/galaxy/galaxy-central/commits/e786022dc67ed918050bd81b9ac679ac958e4f75 is in your distribution and if it is try

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread John Chilton
Looks to yes... % hg contains -c c458a0fe1ba8 e786022dc67ed918050bd81b9ac679ac958e4f75 lookup for revision 'e786022dc67ed918050bd81b9ac679ac958e4f75' yields changeset 10953:e786022dc67e true You could also verify lib/galaxy/jobs/__init__.py contains the import from galaxy.util import unicodify.

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread David Hoover
Right, nevermind, 'hg log' listed that changeset 10953:e786022dc67e. Changing DEFAULT_ENCODING to 'latin-1' in lib/galaxy/util/__init__.py worked. Do I need to alter ALL the MySQL tables to UTF-8, or just a selection of tables? Will future updates explicitly create new tables with

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread John Chilton
Fantastic! For this particular problem - I guess you don't strictly need to modify more than just job and maybe task tables. I suspect at some point there will be a non-latin-1 job parameter or history name or username, etc... that will result in a similar problem though - so if you could just

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread David Hoover
John, I stopped galaxy, then ran ALTER DATABASE galaxydb DEFAULT CHARACTER SET = 'utf8', then ran ALTER TABLE `[table]` DEFAULT CHARACTER SET = 'utf8' on all the tables in galaxydb. After starting up galaxy and rerunning the jobs (using the unaltered version of lib/galaxy/util/__init__.py),

Re: [galaxy-dev] error with unicode output

2013-12-05 Thread John Chilton
Hmmm... can you try adding ?charset=utf8 to your database connection string - that may fix the problem? If not - is there a way to tell if the actual columns have changed. Some comments on stackoverflow make it sound like the commands you listed will only affect new columns. Can you try the