Re: [galaxy-dev] test/base/twilltestcase.py - Converting local (test-data) bam to sam failed
Hey Peter, Thanks for continuing to apply pressure on these issues. I have pushed some logging into Galaxy's development branch (https://github.com/galaxyproject/galaxy/commit/8cb06d7fc2913b4d83ca01b50d76e9607bbe379d) that should make that error more informative. I don't know when that will get pushed out to bitbucket and the shed retested - but once those two things happen we will hopefully have a better idea what the problem is (probably some subtle deployment problem - some incompatible version of samtools on the path? samtools on Galaxy's path but not the test script's path?). -John On Mon, Mar 23, 2015 at 6:06 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Tue, Mar 3, 2015 at 9:55 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Tue, Jan 27, 2015 at 2:23 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I have a query about a failing tool test on the Tool Shed, where it seems Galaxy is trying to convert both the expected BAM output and the tool's BAM output into SAM for comparison - but this fails. https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-27 02:13:51 ... Converting local (test-data) bam to sam failed (and same error on the second test) https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad is still failing, although now with a more concise traceback: Test runs 2015-01-29 02:26:31 Automated test environment Tests that failed Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_00 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/samtools_depad/0.0.1) Stderr: Traceback: Traceback (most recent call last): ... Converting local (test-data) bam to sam failed The same occurs on the Test Tool Shed (with a very slightly updated version of this tool), although with a more concise traceback: https://testtoolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-25 08:21:59 ... Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_00 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/samtools_depad/0.0.2) Stderr: Traceback: ... Converting local (test-data) bam to sam failed https://testtoolshed.g2.bx.psu.edu/view/peterjc/samtools_depad also still failing, Test runs 2015-01-25 08:21:59 ... Converting local (test-data) bam to sam failed Still failing on the Test Tool Shed, bug filed on Trello, https://trello.com/c/TL0IdLlG/2570-toolshed-converting-local-test-data-bam-to-sam-failed Tests currently stalled on the main Tool Shed, reported here: http://dev.list.galaxyproject.org/Tests-not-being-run-on-toolsheds-tc4666816.html https://lists.galaxyproject.org/pipermail/galaxy-dev/2015-March/021708.html Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] value_from_basic - KeyError: 'base_name', was: Tests not being run on toolsheds?
Hi Dave, Now that you've fixed some of the test back log, the bad news is this issue I reported last week appears to be a major regression affecting multiple tools on the Test Tool Shed: https://testtoolshed.g2.bx.psu.edu/view/peterjc/align_back_trans https://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go https://testtoolshed.g2.bx.psu.edu/view/peterjc/blastxml_to_top_descr https://testtoolshed.g2.bx.psu.edu/view/peterjc/clinod ... https://testtoolshed.g2.bx.psu.edu/view/peterjc/nlstradamus ... https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs ... https://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_select_by_id These are all recent test runs from 2015-03-18. Note that some tool tests from the same date are passing, e.g. https://testtoolshed.g2.bx.psu.edu/view/peterjc/blast_rbh https://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_composition https://testtoolshed.g2.bx.psu.edu/view/peterjc/mummer I have not yet spotted any pattern in this division. (However this is clearly not linked to the test expect_failure test I'd added to the sample_seqs tool, as I speculated last week.) Regards, Peter On Wed, Mar 18, 2015 at 2:14 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Dave, The following looks like a regression on the Test Tool Shed, the tests pass locally (using a recent revision), and on TravisCI using the current galaxy dev branch on GitHub: https://travis-ci.org/peterjc/pico_galaxy/builds/54870500 Problem tool: https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs This may be unrelated, but by chance this is the first time I have uploaded a tool to the Tool Shed which uses the new functionality to test the stdout/stderr strings, and more importantly it includes a test expected to fail via test expect_failure=true expect_exit_code=1 Revision on my development repository: https://github.com/peterjc/pico_galaxy/commit/55ebb308b911b4acef912cc3b03f4371f4c6dfe6 Test Tool Shed output from last night: Automated test environment Time tested: 2015-03-18 02:46:55 System: Linux 3.13.0-36-generic Architecture: x86_64 Python version: 2.7.6 Galaxy revision: 17050:6395e7035143 Galaxy database version: 128 Tool shed revision: 16867:0468d285f89c Tool shed database version: 25 Tool shed mercurial version: 3.2.4 Tests that failed Tool id: sample_seqs Tool version: sample_seqs Test: test_tool_00 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/sample_seqs/sample_seqs/0.2.1) Stderr: Traceback: Traceback (most recent call last): File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py, line 268, in test_tool self.do_it( td ) File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py, line 37, in do_it stage_data_in_history( galaxy_interactor, testdef.test_data(), test_history, shed_tool_id ) File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py, line 38, in stage_data_in_history upload_wait() File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py, line 279, in wait while not self.__history_ready( history_id ): File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py, line 297, in __history_ready return self._state_ready( state, error_msg=History in error state. ) File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py, line 356, in _state_ready raise Exception( error_msg ) Exception: History in error state. Traceback (most recent call last): File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/__init__.py, line 158, in prepare_job job_wrapper.prepare() File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/__init__.py, line 828, in prepare tool_evaluator.set_compute_environment( compute_environment, get_special=get_special ) File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/evaluation.py, line 53, in set_compute_environment incoming = self.tool.params_from_strings( incoming, self.app ) File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/__init__.py, line 1873, in params_from_strings return params_from_strings( self.inputs, params, app, ignore_errors ) File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/parameters/__init__.py, line 103, in params_from_strings value = params[key].value_from_basic( value, app, ignore_errors ) File /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/parameters/grouping.py, line 512, in value_from_basic raise e KeyError: 'base_name' requests.packages.urllib3.connectionpool: DEBUG: GET
Re: [galaxy-dev] Ideas: allow Fieldset in tool forms
Hello Gildas, You may want to check out the discussion on this Trello card - https://trello.com/c/KxlQK0FB. This is a frequently requested feature with a lot of votes. I think Aysam is actively working on it - so I would expect it to be in Galaxy soon. -John On Mon, Mar 23, 2015 at 7:33 AM, Gildas Le Corguille lecorgui...@sb-roscoff.fr wrote: Dear Galaxy Team and al., To improve the usability, it would be nice to group parameters in some kinds of fieldset. We can already hide some advanced options with conditional but ... But it's just a personal wishes And as always, great job :) Cheers Gildas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] test/base/twilltestcase.py - Converting local (test-data) bam to sam failed
On Mon, Mar 23, 2015 at 1:41 PM, John Chilton jmchil...@gmail.com wrote: Hey Peter, Thanks for continuing to apply pressure on these issues. I have pushed some logging into Galaxy's development branch (https://github.com/galaxyproject/galaxy/commit/8cb06d7fc2913b4d83ca01b50d76e9607bbe379d) that should make that error more informative. I don't know when that will get pushed out to bitbucket and the shed retested - but once those two things happen we will hopefully have a better idea what the problem is (probably some subtle deployment problem - some incompatible version of samtools on the path? samtools on Galaxy's path but not the test script's path?). -John Lovely - I agree this seems the most likely cause, so fingers crossed. Thank you, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Ideas: allow Fieldset in tool forms
Dear Galaxy Team and al., To improve the usability, it would be nice to group parameters in some kinds of fieldset. We can already hide some advanced options with conditional but ... But it's just a personal wishes And as always, great job :) Cheers Gildas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] wrong validation message in toolshed
I now noticed that another repository I own also only works after I choose “reset all metadata” at Tool Shed. See https://toolshed.g2.bx.psu.edu/view/pieterlukasse/primo_multiomics This repository does not seem to have any problems... Could there be something wrong with Tool Shed and our version of Galaxy (incompatibility issues)? And why would “reset all metadata” on tool shed solve the issue? Now for every commit I make, I seem to have to do this...strange. Thanks, Pieter From: galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] On Behalf Of Lukasse, Pieter Sent: donderdag 19 maart 2015 17:39 To: 'Martin Čech'; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] wrong validation message in toolshed Hi Martin, Thanks. I fixed the problems in the .xml files, now referring to “../match_library.py” . Now the error I get when I try to reset the repository metadata is 502: 502 Bad Gateway nginx/1.2.1 Regards, Pieter From: Martin Čech [mailto:mar...@bx.psu.edu] Sent: donderdag 19 maart 2015 15:02 To: Lukasse, Pieter; galaxy-dev@lists.galaxyproject.orgmailto:galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] wrong validation message in toolshed Hi Pieter, there seems to be either multiple incorrect links to the file or the wrappers expect multiple files that are missing from the repo. E.g. your wrapper library_lookup.xml refers to a file /GCMS/match_library.py which does not exist. (on this line: code file=match_library.py /) After you correct these links you might want to 'Reset repository metadata'. Martin On Thu, Mar 19, 2015 at 8:18 AM Lukasse, Pieter pieter.luka...@wur.nlmailto:pieter.luka...@wur.nl wrote: Hi , I’m getting this error message, which is not correct since the file IS available in the repository. Metadata may have been defined for some items in revision 'f70b775dcaa5'. Correct the following problems if necessary and reset metadata. create_model.xml - This file refers to a missing file match_library.py. Upload a file named match_library.py to the repository to correct this error. library_lookup.xml - This file refers to a missing file match_library.py. Upload a file named match_library.py to the repository to correct this error. query_metexp.xml - This file refers to a missing file match_library.py. Upload a file named match_library.py to the repository to correct this error. query_mass_repos.xml - This file refers to a missing file match_library.py. Upload a file named match_library.py to the repository to correct this error. metams_lcms_annotate.xml - This file refers to a missing file match_library.py. Upload a file named match_library.py to the repository to correct this error. rankfilterGCMS_tabular.xml - This file refers to a missing file match_library.py. Upload a file named match_library.py to the repository to correct this error. metams_lcms_annotate.xml - This file refers to a missing file match_library.py. Upload a file named match_library.py to the repository to correct this error. I installed the package anyway and it works, which proves there is no real problem. See https://toolshed.g2.bx.psu.edu/view/pieterlukasse/prims_metabolomics2 Regards, Pieter Lukasse Wageningen UR, Plant Research International Department of Bioinformatics (Bioscience) Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands T: +31-317481122; M: +31-628189540; skype: pieter.lukasse.wur http://www.pri.wur.nlhttp://www.pri.wur.nl/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] test/base/twilltestcase.py - Converting local (test-data) bam to sam failed
On Tue, Mar 3, 2015 at 9:55 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Tue, Jan 27, 2015 at 2:23 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I have a query about a failing tool test on the Tool Shed, where it seems Galaxy is trying to convert both the expected BAM output and the tool's BAM output into SAM for comparison - but this fails. https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-27 02:13:51 ... Converting local (test-data) bam to sam failed (and same error on the second test) https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad is still failing, although now with a more concise traceback: Test runs 2015-01-29 02:26:31 Automated test environment Tests that failed Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_00 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/samtools_depad/0.0.1) Stderr: Traceback: Traceback (most recent call last): ... Converting local (test-data) bam to sam failed The same occurs on the Test Tool Shed (with a very slightly updated version of this tool), although with a more concise traceback: https://testtoolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-25 08:21:59 ... Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_00 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/samtools_depad/0.0.2) Stderr: Traceback: ... Converting local (test-data) bam to sam failed https://testtoolshed.g2.bx.psu.edu/view/peterjc/samtools_depad also still failing, Test runs 2015-01-25 08:21:59 ... Converting local (test-data) bam to sam failed Still failing on the Test Tool Shed, bug filed on Trello, https://trello.com/c/TL0IdLlG/2570-toolshed-converting-local-test-data-bam-to-sam-failed Tests currently stalled on the main Tool Shed, reported here: http://dev.list.galaxyproject.org/Tests-not-being-run-on-toolsheds-tc4666816.html https://lists.galaxyproject.org/pipermail/galaxy-dev/2015-March/021708.html Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tests not being run on toolsheds?
Hi Dave, Can you check for stalls on the following repositories please? They've not been tested for nearly two months: https://toolshed.g2.bx.psu.edu/view/peterjc/clinod Last tested 2015-01-29, Exception: Job in error state. https://toolshed.g2.bx.psu.edu/view/peterjc/blast2go Last tested 2015-01-29, Exception: Job in error state. https://toolshed.g2.bx.psu.edu/view/peterjc/effectivet3 Last tested 2015-01-29, Exception: Job in error state. https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad Last tested 2015-01-29, Converting local (test-data) bam to sam failed The cause of the last failure is different (and is also failing on the Test Tool Shed), raised separately here: https://lists.galaxyproject.org/pipermail/galaxy-dev/2015-March/021650.html http://dev.list.galaxyproject.org/test-base-twilltestcase-py-Converting-local-test-data-bam-to-sam-failed-tc4666509.html -- Could you tweak the Latest revision: failing tool tests output to include the date last tested in the table of results? Over on the test tool shed there are plenty of recent failures which I will email about separately - I'll report back here if I spot any more stall entries where there tests look backlogged. Thanks, Peter On Wed, Mar 18, 2015 at 1:46 PM, Dave Bouvier d...@bx.psu.edu wrote: Gentlemen, The issue with the nightly testing was due to a stalled test run blocking subsequent tests. I've cleared out that blockage and a manual test run appears to have completed successfully, as should future automated test runs. As always, feel free to let us know if you encounter any additional inexplicable behavior. --Dave B. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
