Re: [galaxy-dev] tool xml substitutes special characters in text parameter

2014-07-28 Thread Peter Cock
On Mon, Jul 28, 2014 at 11:03 AM, Wolfgang Maier
wolfgang.ma...@biologie.uni-freiburg.de wrote:
 Dear all,

 I noticed that with params of type text Galaxy seems to replace certain
 characters before passing them to the shell. As examples, it changes @ to
 __at__, } to __cc__ and \ to X.
 Is this the standard behavior or am I doing something wrong ? And if it's
 standard, are there workarounds ?

 Best,
 Wolfgang

This is standard behaviour to prevent special characters being used
to construct malicious command lines. It can be configured within
your tool definition using the sanitizer tag set:

https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E_tag_set

Peter
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] tool xml substitutes special characters in text parameter

2014-07-28 Thread Wolfgang Maier

On 28.07.2014 12:22, Peter Cock wrote:


This is standard behaviour to prevent special characters being used
to construct malicious command lines. It can be configured within
your tool definition using the sanitizer tag set:

https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E_tag_set

Peter



Thanks a lot, Peter, that solved my problem !

Unfortunately, with this one fixed I now run into an additional one:

There is one free-text text field defined in my tool, which should 
accept characters outside the standard ascii code range (i.e.  127), in 
particular, German Umlaute äöüÄÖÜ.

They are now no longer translated to Xs, but instead I get:

an 'ascii' codec can't decode byte 0xc3 in position 22: ordinal not in 
range(128) error, while parsing the content of the field inside the 
command tag.
Is there any way to make Galaxy work with things outside the ascii range 
(like by using a different codec)?


Just in case, here is the complete traceback I'm seeing:

Traceback (most recent call last):
  File 
/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py, 
line 152, in prepare_job

job_wrapper.prepare()
  File /home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/jobs/__init__.py, 
line 679, in prepare

self.command_line, self.extra_filenames = tool_evaluator.build()
  File 
/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/tools/evaluation.py, line 
323, in build

self.__build_command_line( )
  File 
/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/tools/evaluation.py, line 
339, in __build_command_line

command_line = fill_template( command, context=param_dict )
  File /home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/util/template.py, 
line 9, in fill_template

return str( Template( source=template_text, searchList=[context] ) )
  File 
/home/galaxy/WMGalaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py, 
line 1004, in __str__

return getattr(self, mainMethName)()
  File 
cheetah_DynamicallyCompiledCheetahTemplate_1406545596_05_93125.py, 
line 163, in respond
  File 
/home/galaxy/WMGalaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/DummyTransaction.py, 
line 47, in getvalue

(%s)''' % (nonunicode, ex))
DummyResponseFailure: Looks like you're trying to mix encoded strings 
with Unicode strings

(most likely utf-8 encoded ones)

This can happen if you're using the `EncodeUnicode` filter, 
or if you're manually
encoding strings as utf-8 before passing them in on the 
searchList (possible offenders:
['ascii', 'ascii', ' ', 'non-ASCII characters: 
\xc3\x84\xc3\x96\xc3\x9c', 
'/home/galaxy/WMGalaxy/galaxy-dist/database/files/000/dataset_197.dat'])
('ascii' codec can't decode byte 0xc3 in position 22: 
ordinal not in range(128))


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] tool xml substitutes special characters in text parameter

2014-07-28 Thread Peter Cock
On Mon, Jul 28, 2014 at 12:23 PM, Wolfgang Maier
wolfgang.ma...@biologie.uni-freiburg.de wrote:
 On 28.07.2014 12:22, Peter Cock wrote:

 This is standard behaviour to prevent special characters being used
 to construct malicious command lines. It can be configured within
 your tool definition using the sanitizer tag set:

 https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E_tag_set

 Peter


 Thanks a lot, Peter, that solved my problem !

 Unfortunately, with this one fixed I now run into an additional one:

 There is one free-text text field defined in my tool, which should accept
 characters outside the standard ascii code range (i.e.  127), in
 particular, German Umlaute äöüÄÖÜ.

Hmm. This may be very complicated since a lot will depend on the
local server/cluster's locale settings. Not everything will be UTF-8.

Peter

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/