Re: [galaxy-dev] Uploading Binary Files into Galaxy
Hi, Have you tried grep'ing the error message in the lib files to see where Galaxy goes to during the upload of these files? like: grep /The uploaded binary file contains inappropriate content/ `find *` I had a similar problem when I modified Galaxy to manage .gz files without uncompressing them automatically (I had to add a new file type too), and diving into the code to see what is actually executed really helped. Good luck, L-A Le 24/05/2011 04:40, darren.culle...@csiro.au a écrit : Hi All, Sorry for the delay, I have had a couple of other projects on the go. Still no good. I have made the data type unsniffable and added what I thought was the correct information. I have also added the extension (sfx) to the list of unsniffable binary formats in binary.py but it does not make a difference. Code is as follows: ./datatypes_conf.xml datatype extension=sfx type=galaxy.datatypes.binary:Sfx display_in_upload=true/ ./lib/galaxy/datatypes/binary.py class Sfx( Binary ): Kanga and uAssembler Suffix Array file_ext = sfx def __init__( self, **kwd ): Binary.__init__( self, **kwd ) def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: dataset.peek = 'Suffix Array sfx' dataset.blurb = 'binary data' else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' My gut feeling is that this is related to the fact that Galaxy wants to look in the file, but as soon as its opened, it can't recognise the characters and it reports /The uploaded binary file contains inappropriate content/. I thought the set_peekmethod I have above would override the one described in the Class Binary. I have tried adding a sniffer tag into datatypes_conf.xml, but I get the same result whether the sniffer tag is there or not. I am sure I have missed a step or piece of content but I have no idea what I am missing... Thanks again for your time, Darren Cullerne Computational Biology CSIRO Plant Industry x5042 +61 2 62465042 darren.culle...@csiro.au mailto:darren.culle...@csiro.au ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Uploading Binary Files into Galaxy
darren.culle...@csiro.au wrote: Hi All, Sorry for the delay, I have had a couple of other projects on the go. Still no good. I have made the data type unsniffable and added what I thought was the correct information. I have also added the extension (sfx) to the list of unsniffable binary formats in binary.py but it does not make a difference. Hi Darren, Please have a look in tools/data_source/upload.py, you'll need to bypass the binary test for your new datatype. Eventually this test should be moved up to the datatype code, but for now this extra step is required. --nate Code is as follows: ./datatypes_conf.xml datatype extension=sfx type=galaxy.datatypes.binary:Sfx display_in_upload=true/ ./lib/galaxy/datatypes/binary.py class Sfx( Binary ): Kanga and uAssembler Suffix Array file_ext = sfx def __init__( self, **kwd ): Binary.__init__( self, **kwd ) def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: dataset.peek = 'Suffix Array sfx' dataset.blurb = 'binary data' else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' My gut feeling is that this is related to the fact that Galaxy wants to look in the file, but as soon as its opened, it can't recognise the characters and it reports The uploaded binary file contains inappropriate content. I thought the set_peek method I have above would override the one described in the Class Binary. I have tried adding a sniffer tag into datatypes_conf.xml, but I get the same result whether the sniffer tag is there or not. I am sure I have missed a step or piece of content but I have no idea what I am missing... Thanks again for your time, Darren Cullerne Computational Biology CSIRO Plant Industry x5042 +61 2 62465042 darren.culle...@csiro.aumailto:darren.culle...@csiro.au ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Uploading Binary Files into Galaxy
Hi All, Sorry for the delay, I have had a couple of other projects on the go. Still no good. I have made the data type unsniffable and added what I thought was the correct information. I have also added the extension (sfx) to the list of unsniffable binary formats in binary.py but it does not make a difference. Code is as follows: ./datatypes_conf.xml datatype extension=sfx type=galaxy.datatypes.binary:Sfx display_in_upload=true/ ./lib/galaxy/datatypes/binary.py class Sfx( Binary ): Kanga and uAssembler Suffix Array file_ext = sfx def __init__( self, **kwd ): Binary.__init__( self, **kwd ) def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: dataset.peek = 'Suffix Array sfx' dataset.blurb = 'binary data' else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' My gut feeling is that this is related to the fact that Galaxy wants to look in the file, but as soon as its opened, it can't recognise the characters and it reports The uploaded binary file contains inappropriate content. I thought the set_peek method I have above would override the one described in the Class Binary. I have tried adding a sniffer tag into datatypes_conf.xml, but I get the same result whether the sniffer tag is there or not. I am sure I have missed a step or piece of content but I have no idea what I am missing... Thanks again for your time, Darren Cullerne Computational Biology CSIRO Plant Industry x5042 +61 2 62465042 darren.culle...@csiro.aumailto:darren.culle...@csiro.au ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Uploading Binary Files into Galaxy
Hi All, Sorry for the delay, I have had a couple of other projects on the go. Still no good. I have made the data type unsniffable and added what I thought was the correct information. I have also added the extension (sfx) to the list of unsniffable binary formats in binary.py but it does not make a difference. Code is as follows: ./datatypes_conf.xml datatype extension=sfx type=galaxy.datatypes.binary:Sfx display_in_upload=true/ ./lib/galaxy/datatypes/binary.py class Sfx( Binary ): Kanga and uAssembler Suffix Array file_ext = sfx def __init__( self, **kwd ): Binary.__init__( self, **kwd ) def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: dataset.peek = 'Suffix Array sfx' dataset.blurb = 'binary data' else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' My gut feeling is that this is related to the fact that Galaxy wants to look in the file, but as soon as its opened, it can't recognise the characters and it reports The uploaded binary file contains inappropriate content. I thought the set_peek method I have above would override the one described in the Class Binary. I have tried adding a sniffer tag into datatypes_conf.xml, but I get the same result whether the sniffer tag is there or not. I am sure I have missed a step or piece of content but I have no idea what I am missing... Thanks again for your time, Darren Cullerne Computational Biology CSIRO Plant Industry x5042 +61 2 62465042 darren.culle...@csiro.aumailto:darren.culle...@csiro.au ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Uploading Binary Files into Galaxy
Hi In your_galaxy_root_path/lib/galaxy/datatypes/binary.py try adding the extension of your suffix array file type in the unsniffable_binary_formats array. It would be a good idea to add a class for this file type here too and make sure the datatypes_conf.xml file refers to it. Hope that helps L-A darren.culle...@csiro.au wrote: Hi Everyone, I have encountered a problem and I don’t know how to fix it. We have built a Galaxy instance on a couple of machines here and have built the XML files to use a couple of tools in Galaxy. The input of one of these is a Suffix Array (a binary file). Unfortunately, when I try and upload the Suffix Array, I get the following error message: /The uploaded binary file contains inappropriate content/ My gut feeling is that it is related to Galaxy trying to preview the file and not being able to determine the type. Is there any way around this? I can provide the file and the tool if you need to investigate it further. Thanks, Darren Cullerne Computational Biology CSIRO Plant Industry x5042 +61 2 62465042 darren.culle...@csiro.au mailto:darren.culle...@csiro.au ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Uploading Binary Files into Galaxy
Hi Darren I assume, you have created a new data type https://bitbucket.org/galaxy/galaxy-central/wiki/AddingDatatypes You can now disable the 'sniffing' by adding the data type to the following line: unsniffable_binary_formats = [ 'ab1', 'scf' ] in: ~/galaxy-dist/lib/galaxy/datatypes/binary.py I hope this helps Regards, Hans On 05/06/2011 03:59 AM, darren.culle...@csiro.au wrote: Hi Everyone, I have encountered a problem and I don't know how to fix it. We have built a Galaxy instance on a couple of machines here and have built the XML files to use a couple of tools in Galaxy. The input of one of these is a Suffix Array (a binary file). Unfortunately, when I try and upload the Suffix Array, I get the following error message: The uploaded binary file contains inappropriate content My gut feeling is that it is related to Galaxy trying to preview the file and not being able to determine the type. Is there any way around this? I can provide the file and the tool if you need to investigate it further. Thanks, Darren Cullerne Computational Biology CSIRO Plant Industry x5042 +61 2 62465042 darren.culle...@csiro.aumailto:darren.culle...@csiro.au ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
