Re: [galaxy-dev] How to remove a broken toolshed install
Hi Greg, Thanks for this! On 17 October 2012 01:17, Greg Von Kuster g...@bx.psu.edu wrote: I managed to break a toolshed-installed tool by fiddling with the files under shed_tools. As you've discovered, this is not a good thing to try. Always use the Galaxy interface features to perform tasks like this. Actually, the reason I did this was because I didn't know how to solve a different problem, so maybe I should ask you about that one as well. We had some occasions where we'd try to install a tool from the toolshed, and it would hang - it appeared that the hg pull was timing out. In these cases the config files wouldn't get set up, but a partial repository was pulled / directories were created, and the repository files would then get in the way of trying to install the tool (it seemed to think it was already there). The only way to fix it seemed to be to manually delete the partially-pulled repository under shed_tools. This worked fine for fixing failed installs. But, this time, I thought (wrongly) that this had happened again and I deleted a repository - then realised that it was actually installed and registered in the database, etc. So, if the hg pull times out, is there a right way to clean up the resulting files? I got in the habit of doing it manually, which of course is dangerous, because I didn't know any way to do it via the admin interface. Depending on the changes you've made, you should be able to do the following: 1. Manually remove the installed repository subdirectory hierarchy from disk. 2. If the repository included any tools, manually remove entries for each of them from the shed_tool-conf.xml file ( or the equivalent file you have configured for handling installed repositories ) 3. Manually update the database using the following command (assuming your installed repository is named 'bcftools_view' and it is the only repository you have installed with that name) - letter capitalization is required: The following assumes you're using postgres: update tool_shed_repository set deleted=True, uninstalled=True, status='Uninstalled', error_message=Null where name = 'bcftools_view'; Thanks very much! Yes it's postgres. I'll let you know if I succeed. Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Display of HTML type broken in new Galaxy ?
Hi, I noticed that in the new Galaxy version the preview of HTML files seems to be broken. When I try to preview a HTML file generated by a tool, it displays a HTML that has all style stripped off. So the HTML display is in fact changing the generated HTML before displaying it. Has anyone noticed this as well? What are the fixes/workarounds? [cid:image001.jpg@01CDAC4F.72D92640] Regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nlhttp://www.pri.wur.nl/ inline: image001.jpg___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Display of HTML type broken in new Galaxy ?
Hi, Make sure that in your universe.wsgi is set: sanitize_all_html = False Hope this helps, Joachim Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 10/17/2012 10:09 AM, Lukasse, Pieter wrote: Hi, I noticed that in the new Galaxy version the preview of HTML files seems to be broken. When I try to preview a HTML file generated by a tool, it displays a HTML that has all style “stripped off”. So the HTML display is in fact changing the generated HTML before displaying it. Has anyone noticed this as well? What are the fixes/workarounds? Regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nl http://www.pri.wur.nl/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] renaming fastq to fastqsanger
Hi, I have a similar problem with renaming fastq to fastqsanger. In my case after some time galaxy displays An error occurred setting the metadata for this dataset. You may be able to set it manually or retry auto-detection. The paster.log shows this: galaxy.jobs DEBUG 2012-10-17 14:16:44,623 (123) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /data/galaxy/galaxy-dist/database/job_working_directory/000/123 galaxy 3400 galaxy.jobs DEBUG 2012-10-17 14:16:44,989 Tool did not define exit code or stdio handling; checking stderr for success 141.80.188.178 - - [17/Oct/2012:14:16:45 +0200] POST /galaxy/root/history_item_updates HTTP/1.1 200 - http://bbc.mdc-berlin.de/galaxy/history?dataset_id=f30a35c999095ed7filename=None; Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0 galaxy.datatypes.metadata DEBUG 2012-10-17 14:16:45,234 setting metadata externally failed for HistoryDatasetAssociation 110: External set_meta() not called galaxy.jobs DEBUG 2012-10-17 14:16:45,375 job 123 ended May that be related to an incorrect setup of drmaa job-runner? regards, Andreas On 03.10.2012 18:49, Pete Schmitt wrote: When attempting the change the filetype of a fastq file to fastqsanger, the process for changing the type goes on forever. This does not allow any further processing of the data. -- * Pete Schmitt Technical Director: Discovery Cluster NH INBRE Grid Computational Genetics Lab Dartmouth College, HB 6223 L24 Berry/Baker Library Hanover, NH 03755 Phone: 603-646-8109 http://discovery.dartmouth.edu http://columbia.dartmouth.edu/grid http://www.epistasis.org * This body part will be downloaded on demand. -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Which bam_to_bigwig to use
In the main toolshed I find bam_to_bigwig 2 times. Description, Version and Requirements seem the same. How am I to decide which one to use? regards, Andreas -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error when creating first data library in new Galaxy installation
For info: turns out this was already reported and fixed back on the 5th October in galaxy-central changeset 210c39f4bf7f, specifically applying these patches: https://bitbucket.org/galaxy/galaxy-central/changeset/029818b7e546a3b6835c49ef4f8694c5ac695dfb fixed the problem for me. (Thanks Scott McManus for the original fix.) Best wishes, Peter On 16/10/12 16:36, Peter Briggs wrote: Hello I've encountered an error when creating the first data library within a Galaxy installation. The data library appears to be created but an error message appears (Error attempting to display contents of library (Test Data): No module named controllers.library_common - see attached screenshot) and although the library name appears, it's not possible to interact further with the library e.g. to add datasets. The original error occurred on our nascent production setup, but I can reproduce it on a brand new vanilla local galaxy instance (changeset 7828:b5bda7a5c345) with the only configuration being to register as a user and set the admin_users parameter in universe_wsgi.ini. Then, go to admin - manage data libraries - create new data library. This error doesn't appear when creating a new data library on an instance which already has existing libraries, and debug output doesn't give me any additional clues about where the missing module is coming from. I don't think this problem has been reported previously however apologies if it's already been addressed on the mailing list. Thanks for any help Best wishes, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Which bam_to_bigwig to use
On Wed, Oct 17, 2012 at 1:30 PM, Andreas Kuntzagk andreas.kuntz...@mdc-berlin.de wrote: In the main toolshed I find bam_to_bigwig 2 times. Description, Version and Requirements seem the same. How am I to decide which one to use? regards, Andreas A little history: Brad's was first, using pysam and wigToBigWig, and could be run on a standalone BAM file, but initially didn't allow ignoring of CIGAR N operations. Lance wrote an alternative using using genomeCoverageBed and bedGraphToBigWig allowing gaps/skips to be excluded from the coverage calculation, which is important for RNA-Seq. However, it did not work on standalone BAM files (e.g. reads mapped to an in house assembly), only those with a genome defined in Galaxy. I helped combine them to produce the updated version of Brad's tool, which works on standalone BAM files, and can ignore CIGAR gaps/ skips. So personally we the latest version under 'brad-chapman'. For more details, see this thread (and follow ups later in the year). http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-April/009403.html (Brad and Lance - please correct me if I've misrepresented anything in that summary) As this point it might be nice to make the gap/skip handling an option in Brad's wrapper (defaulting to on to match the current behaviour). Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] the comtent syntax of command and configfile
Hi, In the command tag of the tools/update.xml file, the paramfile was setted by the result of some statements followed, I want to know what syntax is and if it is OK that I move the statements to configfile tag? Thanks, welkin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Number of outputs = number of inputs
Like most days, JJ very politely pointed out that I am a wrong this morning. You can have variable numbers of outputs at runtime, see the last section (Number of Output datasets cannot be determined until tool run ) of this page: http://wiki.g2.bx.psu.edu/Admin/Tools/Multiple%20Output%20Files Sorry about that. -John On Tue, Oct 16, 2012 at 8:48 AM, John Chilton chil0...@umn.edu wrote: I don't believe this is possible in Galaxy right now. Are the outputs independent or is information from all inputs used to produce all outputs? If they are independent, you can create a workflow containing just your tool with 1 input and 1 output and use the batch workflow mode to run it on multiple files and get multiple outputs. This is not a beautiful solution but it gets the job done in some cases. Another thing to look at might be the discussion we are having on the thread pass more information on a dataset merge. We have a fork (its all work from Jorrit Boekel) of galaxy that creates composite datatypes for each explicitly defined type that can hold collections of a single type. https://bitbucket.org/galaxyp/galaxy-central-homogeneous-composite-datatypes/compare This would hopefully let you declare that you can accept a collection of whatever your input type is and produce a collection of whatever your output is. Lots of downsides to this approach - not fully implemented, and not included in Galaxy proper, your outputs would be wrapped up in a composite datatype so they wouldn't be easily processable by downstream tools. It would be good to have additional people hacking on it though :) -John John Chilton Senior Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223 Bitbucket: https://bitbucket.org/jmchilton Github: https://github.com/jmchilton Web: http://jmchilton.net On Tue, Oct 16, 2012 at 7:13 AM, Sascha Kastens s.kast...@gatc-biotech.com wrote: Hi all! I have a tool which takes one ore more input files. For each input file one output is created, i.e. 1 input file - 1 output file, 2 input files - 2 output files, etc. What is the best way to handle this? I used the directions for handlin multiple output files where the ’Number of Output datasets cannot be determined until tool run’ which in my opinion is a bit inappropriate. BTW: The input files are added via the repeat-Tag, so maybe there is a similar thing for outputs? Thanks in advance! Cheers, Sascha ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Jobs crash only in workflow context
Todd, this is definitely unusual. Can you post (or send directly) relevant sections from the Galaxy log? -- jt On Tue, Oct 16, 2012 at 8:15 PM, Todd Oakley todd.oak...@lifesci.ucsb.edu wrote: Hello, We just did a few tweaks to improve Galaxy performance, and a new issue popped up that I would like advice on troubleshooting. When we run workflows, we see that tools later in the workflow run and crash before the results they depend on have completed running. We can re-run the crashed jobs later and they work fine, suggesting that they are only failing in the context of running workflows. I'd appreciate any advice on how to start troubleshooting this problem. Thanks much! Todd -- *** Todd Oakley, Professor Ecology Evolution and Marine Biology University of California, Santa Barbara Santa Barbara, CA 93106 USA *** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Odd issue with updateucsc.dh
I had an odd error with wigToBigWig that I tracked back to the *.len files generated by the updateucsc.sh script. With my last UCSC update, a majority of the assemblies have this as the single line: META http-equiv=Content-Script-Type content=text/javascript but others, like Zea_mays_B73_RefGen_v2, are fine. Rerunning this gives the same problem. Any clues? chris ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] error uploading BAM file
Hi, I was trying to upload a 43Gb BAM file to Galaxy when I encountered this problem. I am new to galaxy and python. I have tried uploading files in other formats and they have worked fine. I am using galaxy as a local installation and appreciate any thoughts on how to fix this. Thanks Sukha === Traceback (most recent call last): File /Data/galaxy/galaxy-dist/tools/data_source/upload.py, line 384, in module __main__() File /Data/galaxy/galaxy-dist/tools/data_source/upload.py, line 373, in __main__ add_file( dataset, registry, json_file, output_path ) File /Data/galaxy/galaxy-dist/tools/data_source/upload.py, line 312, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File /Data/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py, line 113, in dataset_content_needs_grooming version = self._get_samtools_version() File /Data/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py, line 97, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File /Data/galaxy/Python-2.7.1/Lib/subprocess.py, line 672, in __init__ errread, errwrite) File /Data/galaxy/Python-2.7.1/Lib/subprocess.py, line 1202, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory == Report this error to the Galaxy Team The Galaxy team regularly reviews errors that occur in the application. However, if you would like to provide additional information (such as what you were trying to do when the error occurred) and a contact e-mail address, we will be better able to investigate your problem and get back to you. ___ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] How to remove a broken toolshed install
Hello Clare, On Oct 17, 2012, at 2:22 AM, Clare Sloggett wrote: Actually, the reason I did this was because I didn't know how to solve a different problem, so maybe I should ask you about that one as well. We had some occasions where we'd try to install a tool from the toolshed, and it would hang - it appeared that the hg pull was timing out. Was the timeout a regular occurrence? if so, do you know that cause, and were you able to get it resolved? In these cases the config files wouldn't get set up, but a partial repository was pulled / directories were created, and the repository files would then get in the way of trying to install the tool (it seemed to think it was already there). The only way to fix it seemed to be to manually delete the partially-pulled repository under shed_tools. The October 5, 2012 Galaxy distribution news brief includes the following link to information about the process for handling repository installation errors, specifically when the errors occur during cloning. http://wiki.g2.bx.psu.edu/InstallingRepositoriesToGalaxy#Handling_repository_installation_errors If you're running an older version of Galaxy, you'll need to update to the October 5 release in order to have these features. The news brief release information is: upgrade: $ hg pull -u -r b5bda7a5c345 Let me know if this is not what you're looking for. Thanks, Greg Von Kuster This worked fine for fixing failed installs. But, this time, I thought (wrongly) that this had happened again and I deleted a repository - then realised that it was actually installed and registered in the database, etc. So, if the hg pull times out, is there a right way to clean up the resulting files? I got in the habit of doing it manually, which of course is dangerous, because I didn't know any way to do it via the admin interface. Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] problem viewing galaxy menu items using IE, version number
Hello, 1. When we access galaxy (local installation) using IE (internet explorer) 8 and 9, we cannot expand/collapse uploaded data files visible from history window. Neither are the info and save icons visible. How do I fix this ? Also items under the main menu are not clickable (e.g. when I click on User menu item, it expands to show User and Register, but I cannot select User or Register items). I can view/click these items using Firefox browser. 2. How do I determine the version number of my installation ? I do not have the original tar file that has the version name. Appreciate your help. Sukha ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [Patch] ToolShed support for included tools
Hello Bjorn, From your comments, it seems you are looking for a way to define a tool dependency that is included in the contents of the repository when it is installed from the tool shed. If this is the case, then your tool_dependencies.xml tag set needs to be changed to the following. tool_dependency set_environment version=1.0 environment_variable name=PATH action=prepend_to$REPOSITORY_INSTALL_DIR/environment_variable /set_environment /tool_deppendency All of the details for how to define your tool configs and dependencies included in your repository like this are available in the following tool shed wiki page. http://wiki.g2.bx.psu.edu/ToolShedToolFeatures#Finding_dependencies_included_in_an_installed_repository_at_tool_execution_time Thanks very much for contributing - I'll take a look at this patch and integrate part or all of it into the next release if appropriate. Greg Von Kuster On Oct 15, 2012, at 1:15 PM, Björn Grüning wrote: Hi, i'm writing a galaxy wrapper for bismark and trim-galore. Both are plain perl scripts, that wraps around other dependencies (e.g. Bowtie). The idea was to include the perl-scripts directly in the galaxy-wrapper and update the PATH to the REPOSITORY_INSTALL_DIR in the tool_dependency.xml file. package name=bismark version=0.7.7 install version=1.0 actions action type=set_environment environment_variable name=PATH action=prepend_to$REPOSITORY_INSTALL_DIR/environment_variable /action /actions /install readme bismark, bismark_genome_preparation and bismark_methylation_extractor are shipped with that wrapper /readme /package Unfortunately, that was not supported because the toolshed expected at least one action_type. The attached patch should add that feature. Furthermore, bowtie2 is only available as zip archive and afaik that was not handled in the toolshed. The attached patch also added check_zipfile(), extract_zip() and zip_extraction_directory() to fully support zip archives. Thanks! Bjoern toolshed_zip.patch___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] How to get a shedtool to run locally
On Mon, Oct 15, 2012 at 11:49 AM, Liisa Koski liisa.ko...@basf.com wrote: Hello, I've installed a tool to my local Galaxy installation via my local ToolShed. I would like to run this tool locally, and not have it submitted to the grid. I have tried two ids in the universe_wsgi.ini [galaxy:tool_runners] section. The tool id directly from the tool.xml file and the id from the shed_tool_conf.xml (same id found in the integrated_tool_panel.xml) tool = local:/// galaxy.server:9009/repos/user/tool/1.0 = local:/// My job still gets submitted to the grid. Hi, I would also like to specify a tool_runner for a tool installed from the shed_tool. As Liisa, I can not find the correct tool id to use. Any help? Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
