[galaxy-dev] Galaxy Security Vulnerability

2014-07-31 Thread Nate Coraor
A security vulnerability was recently discovered by Inge Alexander Raknes that 
would allow a malicious person to execute arbitrary code on a Galaxy server. 
The vulnerability was in a method that uses Python "pickle" functionality to 
decode state information from tool forms. Because pickles can be used to 
instantiate arbitrary Python objects, tool states could be constructed to 
exploit this vulnerability.

Because this vulnerability allows for arbitrary code execution, administrators 
are strongly encouraged to apply this fix IMMEDIATELY. We have tried to make it 
as easy and quick as possible for server administrators to update their Galaxy 
instances. The fix has been applied to every stable release from 2013.01.13 
until the tip, so it is possible to get this fix on older releases without 
updating to a newer feature release. You can do this by identifying your 
current release and updating to the `latest_` tag corresponding 
to your release.

For example, if you are running release_2013.11.04 (or a subsequent commit to 
the stable branch of Galaxy between release_2013.11.04 and release_2014.02.10), 
you can update with:

 % hg pull
 % hg update latest_2013.11.04

For the changes to take effect, YOU MUST RESTART ALL GALAXY SERVER PROCESSES.


If you do not want to pull any upstream changes, we have also created a 
standalone patch that fixes this problem, with multiple versions depending on 
your current Galaxy release:

 - pickle-2013.11.04.patch - This patch should apply cleanly (with offset/fuzz) 
to releases from 2013.11.04 up to the current stable tip. Available at: 
https://depot.galaxyproject.org/patch/pickle-2013.11.04.patch

 - pickle-2013.01.13.patch - This patch should apply cleanly (with offset/fuzz) 
to releases from 2013.01.13 up to 2013.08.12, and possibly older versions of 
Galaxy as well. Available at: 
https://depot.galaxyproject.org/patch/pickle-2013.01.13.patch

 If you happen to be running a very recent revision on the default branch or 
the newly created next-stable branch, a pickle-default.patch file exists at the 
same place.

For older releases or instances with conflicting local modifications, manual 
application of the patch should not be difficult as it only includes a few 
small changes. To apply the patch, navigate to the root of your Galaxy 
directory, then run (replacing  with the URL above that is 
correct for your release):

 % wget -O pickle.patch 

or:

 % curl -o pickle.patch 

and then:

 % patch -p1 < pickle.patch
 patching file lib/galaxy/util/__init__.py
 Hunk #1 succeeded at 575 with fuzz 2 (offset -113 lines).
 patching file lib/galaxy/webapps/galaxy/controllers/ucsc_proxy.py

Again, for the changes to take effect, YOU MUST RESTART ALL GALAXY SERVER 
PROCESSES.


The Galaxy Team would like to extend special thanks to Inge Alexander Raknes 
and colleagues, who privately disclosed the vulnerability, with a full analysis 
and proof of concept.

Credit for the fix and subsequent testing goes to my fellow Galaxy Team members 
John Chilton and Dannon Baker.

On behalf of the Galaxy Team,
--nate___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] package_r_2_11_0 and ggplot2

2014-07-31 Thread Evan Bollig
Btw, I forgot: plyr_1.7.1 is also required for package_gatk_1_4. Hence
the install_url call for it below.

-E
-Evan Bollig
Research Associate | Application Developer | User Support Consultant
Minnesota Supercomputing Institute
599 Walter Library
612 624 1447
e...@msi.umn.edu
boll0...@umn.edu


On Thu, Jul 31, 2014 at 1:51 PM, Björn Grüning
 wrote:
> Hi Evan,
>
> unfortunately I have no access to this repository. I'm only the maintainer
> of the galaxytools in github. But I have CC'ed Dave, he is responsible for
> the devteam packages!
>
> Cheers and thanks for sharing your information!
> Bjoern
>
> Am 31.07.2014 um 20:47 schrieb Evan Bollig:
>
>> Bjoern,
>>
>> FYI, I found that package_gatk_1_4 from the main Galaxy toolshed
>> produces Rscript files for VariantRecalibrator and other commands that
>> require ggplot_0.8.8 and reshape_0.8.4. This applies to GATK versions
>> 1.4-20 and 1.4-37.
>>
>> The magic incantation to get a systemwide R v3.1 setup properly is:
>>
>> r <- getOption("repos");
>> r["CRAN"] <- "http://cran.rstudio.com/";;
>> options(repos=r);
>> install.packages(c('devtools','gsalib'));
>> require(devtools);
>>
>> install_url('http://cran.r-project.org/src/contrib/Archive/plyr/plyr_1.7.1.tar.gz');
>>
>> install_url('http://cran.r-project.org/src/contrib/Archive/reshape/reshape_0.8.4.tar.gz');
>>
>> install_url('http://cran.r-project.org/src/contrib/Archive/ggplot2/ggplot2_0.8.8.tar.gz');
>>
>> I'm not sure gsalib is required, but it is better to play it safe IMO.
>>
>> Can you add this to the package_gatk_1_4 documentation to help others
>> along? It would be even better if the package_gatk_1_4 could also
>> install those R libraries itself.
>>
>> Cheers,
>>
>> -E
>> -Evan Bollig
>> Research Associate | Application Developer | User Support Consultant
>> Minnesota Supercomputing Institute
>> 599 Walter Library
>> 612 624 1447
>> e...@msi.umn.edu
>> boll0...@umn.edu
>>
>>
>> On Tue, Jul 29, 2014 at 4:22 PM, Björn Grüning
>>  wrote:
>>>
>>>
>>>
>>> Am 29.07.2014 um 22:45 schrieb Evan Bollig:
>>>
 Hey Bjoern,

 I'm trying to use variant_recalibrator, xy_plot and other tools that
 depend on package_r_2_11_0 and require ggplot2 to be installed. Since
 CRAN support for R 2.11.0 and 2.15.0 is on the out, perhaps its a
 better idea to update the packages the depend on those versions to R
 3.0 or 3.1?
>>>
>>>
>>>
>>> Yes, this would be a better idea!
>>>
>>>
 The variant_recalibrator tool generates an Rscript
 incompatible with modern versions of ggplot2, so this could be a quite
 involved undertaking.
>>>
>>>
>>>
>>> Oh :( ... can we somehow get an old tarball of ggplot2? Have you looked
>>> at
>>> older linux disributions, debian stable for example?
>>>
>>> I found ggplot on download_store ;)
>>>
>>> https://github.com/bgruening/download_store/blob/master/gatk2_R_deps/ggplot2_0.9.3.1.tar.gz
>>>
>>> Is that old enough?
>>>
>>>
 Meanwhile I'll look into the download_store option. You download all
 of the tarballs and install them individually with R CMD INSTALL?
>>>
>>>
>>>
>>> Not really, we have a nice routine in the toolshed for such tasks:
>>> Have a look at:
>>>
>>> https://github.com/bgruening/galaxytools/tree/master/orphan_tool_dependencies/package_deseq2_1_2_10
>>>
>>> Note: the order of the packages are important ...
>>>
>>> Ciao,
>>> Bjoern
>>>
>>>
 Cheers,

 -E
 -Evan Bollig
 Research Associate | Application Developer | User Support Consultant
 Minnesota Supercomputing Institute
 599 Walter Library
 612 624 1447
 e...@msi.umn.edu
 boll0...@umn.edu


 On Tue, Jul 29, 2014 at 2:46 PM, Björn Grüning
  wrote:
>
>
> Hi Evan,
>
> do you have somehow access to this tarball? Than we can put it on
>
> https://github.com/bgruening/download_store
>
> I use that to host R packages to save them for reproducibility.
>
>
>> Can I install another package_r and have packages that depend on
>> package_r_2_11_0 use the newer version?
>
>
>
>
> I don't think so. But that depends very much on your package. Is it not
> possible to use a newer R version? In either case you should save all
> tarballs somehow to archive them!
>
> Cheers,
> Bjoern
>
>> -Evan Bollig
>> Research Associate | Application Developer | User Support Consultant
>> Minnesota Supercomputing Institute
>> 599 Walter Library
>> 612 624 1447
>> e...@msi.umn.edu
>> boll0...@umn.edu
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/
>>
>
>>>
>

___

Re: [galaxy-dev] package_r_2_11_0 and ggplot2

2014-07-31 Thread Björn Grüning

Hi Evan,

unfortunately I have no access to this repository. I'm only the 
maintainer of the galaxytools in github. But I have CC'ed Dave, he is 
responsible for the devteam packages!


Cheers and thanks for sharing your information!
Bjoern

Am 31.07.2014 um 20:47 schrieb Evan Bollig:

Bjoern,

FYI, I found that package_gatk_1_4 from the main Galaxy toolshed
produces Rscript files for VariantRecalibrator and other commands that
require ggplot_0.8.8 and reshape_0.8.4. This applies to GATK versions
1.4-20 and 1.4-37.

The magic incantation to get a systemwide R v3.1 setup properly is:

r <- getOption("repos");
r["CRAN"] <- "http://cran.rstudio.com/";;
options(repos=r);
install.packages(c('devtools','gsalib'));
require(devtools);
install_url('http://cran.r-project.org/src/contrib/Archive/plyr/plyr_1.7.1.tar.gz');
install_url('http://cran.r-project.org/src/contrib/Archive/reshape/reshape_0.8.4.tar.gz');
install_url('http://cran.r-project.org/src/contrib/Archive/ggplot2/ggplot2_0.8.8.tar.gz');

I'm not sure gsalib is required, but it is better to play it safe IMO.

Can you add this to the package_gatk_1_4 documentation to help others
along? It would be even better if the package_gatk_1_4 could also
install those R libraries itself.

Cheers,

-E
-Evan Bollig
Research Associate | Application Developer | User Support Consultant
Minnesota Supercomputing Institute
599 Walter Library
612 624 1447
e...@msi.umn.edu
boll0...@umn.edu


On Tue, Jul 29, 2014 at 4:22 PM, Björn Grüning
 wrote:



Am 29.07.2014 um 22:45 schrieb Evan Bollig:


Hey Bjoern,

I'm trying to use variant_recalibrator, xy_plot and other tools that
depend on package_r_2_11_0 and require ggplot2 to be installed. Since
CRAN support for R 2.11.0 and 2.15.0 is on the out, perhaps its a
better idea to update the packages the depend on those versions to R
3.0 or 3.1?



Yes, this would be a better idea!



The variant_recalibrator tool generates an Rscript
incompatible with modern versions of ggplot2, so this could be a quite
involved undertaking.



Oh :( ... can we somehow get an old tarball of ggplot2? Have you looked at
older linux disributions, debian stable for example?

I found ggplot on download_store ;)
https://github.com/bgruening/download_store/blob/master/gatk2_R_deps/ggplot2_0.9.3.1.tar.gz

Is that old enough?



Meanwhile I'll look into the download_store option. You download all
of the tarballs and install them individually with R CMD INSTALL?



Not really, we have a nice routine in the toolshed for such tasks:
Have a look at:
https://github.com/bgruening/galaxytools/tree/master/orphan_tool_dependencies/package_deseq2_1_2_10

Note: the order of the packages are important ...

Ciao,
Bjoern



Cheers,

-E
-Evan Bollig
Research Associate | Application Developer | User Support Consultant
Minnesota Supercomputing Institute
599 Walter Library
612 624 1447
e...@msi.umn.edu
boll0...@umn.edu


On Tue, Jul 29, 2014 at 2:46 PM, Björn Grüning
 wrote:


Hi Evan,

do you have somehow access to this tarball? Than we can put it on

https://github.com/bgruening/download_store

I use that to host R packages to save them for reproducibility.



Can I install another package_r and have packages that depend on
package_r_2_11_0 use the newer version?




I don't think so. But that depends very much on your package. Is it not
possible to use a newer R version? In either case you should save all
tarballs somehow to archive them!

Cheers,
Bjoern


-Evan Bollig
Research Associate | Application Developer | User Support Consultant
Minnesota Supercomputing Institute
599 Walter Library
612 624 1447
e...@msi.umn.edu
boll0...@umn.edu
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/






___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] package_r_2_11_0 and ggplot2

2014-07-31 Thread Evan Bollig
Bjoern,

FYI, I found that package_gatk_1_4 from the main Galaxy toolshed
produces Rscript files for VariantRecalibrator and other commands that
require ggplot_0.8.8 and reshape_0.8.4. This applies to GATK versions
1.4-20 and 1.4-37.

The magic incantation to get a systemwide R v3.1 setup properly is:

r <- getOption("repos");
r["CRAN"] <- "http://cran.rstudio.com/";;
options(repos=r);
install.packages(c('devtools','gsalib'));
require(devtools);
install_url('http://cran.r-project.org/src/contrib/Archive/plyr/plyr_1.7.1.tar.gz');
install_url('http://cran.r-project.org/src/contrib/Archive/reshape/reshape_0.8.4.tar.gz');
install_url('http://cran.r-project.org/src/contrib/Archive/ggplot2/ggplot2_0.8.8.tar.gz');

I'm not sure gsalib is required, but it is better to play it safe IMO.

Can you add this to the package_gatk_1_4 documentation to help others
along? It would be even better if the package_gatk_1_4 could also
install those R libraries itself.

Cheers,

-E
-Evan Bollig
Research Associate | Application Developer | User Support Consultant
Minnesota Supercomputing Institute
599 Walter Library
612 624 1447
e...@msi.umn.edu
boll0...@umn.edu


On Tue, Jul 29, 2014 at 4:22 PM, Björn Grüning
 wrote:
>
>
> Am 29.07.2014 um 22:45 schrieb Evan Bollig:
>
>> Hey Bjoern,
>>
>> I'm trying to use variant_recalibrator, xy_plot and other tools that
>> depend on package_r_2_11_0 and require ggplot2 to be installed. Since
>> CRAN support for R 2.11.0 and 2.15.0 is on the out, perhaps its a
>> better idea to update the packages the depend on those versions to R
>> 3.0 or 3.1?
>
>
> Yes, this would be a better idea!
>
>
>> The variant_recalibrator tool generates an Rscript
>> incompatible with modern versions of ggplot2, so this could be a quite
>> involved undertaking.
>
>
> Oh :( ... can we somehow get an old tarball of ggplot2? Have you looked at
> older linux disributions, debian stable for example?
>
> I found ggplot on download_store ;)
> https://github.com/bgruening/download_store/blob/master/gatk2_R_deps/ggplot2_0.9.3.1.tar.gz
>
> Is that old enough?
>
>
>> Meanwhile I'll look into the download_store option. You download all
>> of the tarballs and install them individually with R CMD INSTALL?
>
>
> Not really, we have a nice routine in the toolshed for such tasks:
> Have a look at:
> https://github.com/bgruening/galaxytools/tree/master/orphan_tool_dependencies/package_deseq2_1_2_10
>
> Note: the order of the packages are important ...
>
> Ciao,
> Bjoern
>
>
>> Cheers,
>>
>> -E
>> -Evan Bollig
>> Research Associate | Application Developer | User Support Consultant
>> Minnesota Supercomputing Institute
>> 599 Walter Library
>> 612 624 1447
>> e...@msi.umn.edu
>> boll0...@umn.edu
>>
>>
>> On Tue, Jul 29, 2014 at 2:46 PM, Björn Grüning
>>  wrote:
>>>
>>> Hi Evan,
>>>
>>> do you have somehow access to this tarball? Than we can put it on
>>>
>>> https://github.com/bgruening/download_store
>>>
>>> I use that to host R packages to save them for reproducibility.
>>>
>>>
 Can I install another package_r and have packages that depend on
 package_r_2_11_0 use the newer version?
>>>
>>>
>>>
>>> I don't think so. But that depends very much on your package. Is it not
>>> possible to use a newer R version? In either case you should save all
>>> tarballs somehow to archive them!
>>>
>>> Cheers,
>>> Bjoern
>>>
 -Evan Bollig
 Research Associate | Application Developer | User Support Consultant
 Minnesota Supercomputing Institute
 599 Walter Library
 612 624 1447
 e...@msi.umn.edu
 boll0...@umn.edu
 ___
 Please keep all replies on the list by using "reply all"
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

 To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

>>>
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] fastq-mcf parameter bug

2014-07-31 Thread Michael Mason
First time posting so I hope I do this correctly.

I am using galaxy locally and installed the ea-utils fastq-mcf (FastqMcf). When 
using the trimming options (not defaults) the -C option gets set to the -s 
option. I checked the fastq-mcf.xml wrapper and indeed there is an mistake on 
line 42.

 -s $trimming.sampcnt

Should be
 -C $trimming.sampcnt

I hope this helps,
Cheers,
Mike

--CONFIDENTIALITY NOTICE--: The information contained in this email is intended 
for the exclusive use of the addressee and may contain confidential 
information. If you are not the intended recipient, you are hereby notified 
that any form of dissemination of this communication is strictly prohibited. 
www.benaroyaresearch.org
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] TestToolShed failure, Exception: History in error state.

2014-07-31 Thread Peter Cock
On Thu, Jul 31, 2014 at 5:21 PM, bjoern.gruen...@googlemail.com
 wrote:
> Hi Peter,
>
>
> 2014-07-31 10:57 GMT+02:00 Peter Cock :
>
>> Hi Dave,
>>
>> You are right that on closer inspection I've mixed tool_dependencies.xml
>> and repository_dependencies.xml *again*. Evidentially my mental model
>> does not match Greg's here:
>>
>> (*) I need to define a tool installation recipe for something not in the
>> Tool Shed --> write an install script called tool_dependencies.xml
>>
>> (*) I need to depend on a Python package by pointing at another repository
>> in the Tool Shed --> repository_dependencies.xml
>
> I might be wrong, but I think that also goes to tool_dependencies.xml

Correct, e.g.
https://github.com/peterjc/pico_galaxy/tree/master/tools/seq_select_by_id

Thanks!

>>
>> (*) I need to depend on a datatype package by pointing at another
>> repository in the Tool Shed --> repository_dependencies.xml
>>
>> (*) I need to depend on a binary package by pointing at another repository
>> in the Tool Shed --> repository_dependencies.xml ? No. You need
>> tool_dependencies.xml for this too
>
> As far as I understood, everything that is referenced in the tool.xml under
> the requirement section, needs to be in a tool_dependencies.xml file. Any
> other dependency are from the repository (data_types, data_manager,
> workflows ...).
>
> Ciao,
> Bjoern

Sure, there is a logic here - but its a definition which I seem to still
struggle with :(

>> But that aside, the test framework error here is completely unhelpful.
>>
>> Why is there no error message about missing a dependency?
>> Was there an error from running my tool which was not shown?
>>
>> Thanks,
>>
>> Peter

I'd still like to get a more explicit error from the test suite than
"History in error state" though ;)

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] TestToolShed failure, Exception: History in error state.

2014-07-31 Thread bjoern.gruen...@googlemail.com
Hi Peter,


2014-07-31 10:57 GMT+02:00 Peter Cock :

> Hi Dave,
>
> You are right that on closer inspection I've mixed tool_dependencies.xml
> and repository_dependencies.xml *again*. Evidentially my mental model
> does not match Greg's here:
>
> (*) I need to define a tool installation recipe for something not in the
> Tool Shed --> write an install script called tool_dependencies.xml
>
> (*) I need to depend on a Python package by pointing at another repository
> in the Tool Shed --> repository_dependencies.xml
>

I might be wrong, but I think that also goes to tool_dependencies.xml


> (*) I need to depend on a datatype package by pointing at another
> repository
> in the Tool Shed --> repository_dependencies.xml
>
> (*) I need to depend on a binary package by pointing at another repository
> in the Tool Shed --> repository_dependencies.xml ? No. You need
> tool_dependencies.xml for this too
>


As far as I understood, everything that is referenced in the tool.xml under
the requirement section, needs to be in a tool_dependencies.xml file. Any
other dependency are from the repository (data_types, data_manager,
workflows ...).

Ciao,
Bjoern



> But that aside, the test framework error here is completely unhelpful.
>
> Why is there no error message about missing a dependency?
> Was there an error from running my tool which was not shown?
>
> Thanks,
>
> Peter
>
> On Wed, Jul 30, 2014 at 6:07 PM, Dave Bouvier  wrote:
> > Peter,
> >
> > I believe part of the problem is that the install and test framework is
> > unable to resolve the dependency on blast+ 2.2.29 because it is defined
> as a
> > repository dependency, not a tool dependency. I would recommend replacing
> > the repository dependency in the blast_rbh repository with a tool
> dependency
> > definition that references package_blast_plus_2_2_29.
> >
> > --Dave B.
> >
> > On 07/30/2014 05:27 AM, Peter Cock wrote:
> >>
> >> Hi all,
> >>
> >> I'm not sure when this started (having hardly looked at my Tool Shed
> >> test results since GCC2014), but I think this is a fairly recent problem
> >> with my BLAST RBH tests failing (which has held me back from posting
> >> this to the main Tool Shed).
> >>
> >> This could be some silly mistake in my tar-ball, but usually missing
> >> test files and the like get an explicit error. The tests are passing
> >> on my GitHub/TravisCI setup (using Twill and the API backend):
> >> e.g. https://travis-ci.org/peterjc/galaxy_blast/builds/30592097
> >>
> >> Here is the current error (the same for the last few Test Tool Shed
> >> runs), https://testtoolshed.g2.bx.psu.edu/view/peterjc/blast_rbh
> >>
> >> Traceback (most recent call last):
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
> >> line 108, in test_tool
> >>  self.do_it( td )
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
> >> line 35, in do_it
> >>  self._verify_outputs( testdef, test_history, shed_tool_id,
> >> data_list, galaxy_interactor )
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
> >> line 69, in _verify_outputs
> >>  galaxy_interactor.verify_output( history, output_data,
> >> output_testdef=output_testdef, shed_tool_id=shed_tool_id,
> >> maxseconds=maxseconds )
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
> >> line 53, in verify_output
> >>  self.wait_for_history( history_id, maxseconds )
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
> >> line 107, in wait_for_history
> >>  self.twill_test_case.wait_for( lambda: not self.__history_ready(
> >> history_id ), maxseconds=maxseconds)
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py",
> >> line 2453, in wait_for
> >>  result = func()
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
> >> line 107, in 
> >>  self.twill_test_case.wait_for( lambda: not self.__history_ready(
> >> history_id ), maxseconds=maxseconds)
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
> >> line 257, in __history_ready
> >>  return self._state_ready( state, error_msg="History in error
> state."
> >> )
> >>File
> >>
> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
> >> line 316, in _state_ready
> >>  raise Exception( error_msg )
> >> Exception: History in error state.
> >>
> >> Is a more detailed log available which might help debug this?

Re: [galaxy-dev] TestToolShed failure, Exception: History in error state.

2014-07-31 Thread Peter Cock
Hi Dave,

You are right that on closer inspection I've mixed tool_dependencies.xml
and repository_dependencies.xml *again*. Evidentially my mental model
does not match Greg's here:

(*) I need to define a tool installation recipe for something not in the
Tool Shed --> write an install script called tool_dependencies.xml

(*) I need to depend on a Python package by pointing at another repository
in the Tool Shed --> repository_dependencies.xml

(*) I need to depend on a datatype package by pointing at another repository
in the Tool Shed --> repository_dependencies.xml

(*) I need to depend on a binary package by pointing at another repository
in the Tool Shed --> repository_dependencies.xml ? No. You need
tool_dependencies.xml for this too.

But that aside, the test framework error here is completely unhelpful.

Why is there no error message about missing a dependency?
Was there an error from running my tool which was not shown?

Thanks,

Peter

On Wed, Jul 30, 2014 at 6:07 PM, Dave Bouvier  wrote:
> Peter,
>
> I believe part of the problem is that the install and test framework is
> unable to resolve the dependency on blast+ 2.2.29 because it is defined as a
> repository dependency, not a tool dependency. I would recommend replacing
> the repository dependency in the blast_rbh repository with a tool dependency
> definition that references package_blast_plus_2_2_29.
>
> --Dave B.
>
> On 07/30/2014 05:27 AM, Peter Cock wrote:
>>
>> Hi all,
>>
>> I'm not sure when this started (having hardly looked at my Tool Shed
>> test results since GCC2014), but I think this is a fairly recent problem
>> with my BLAST RBH tests failing (which has held me back from posting
>> this to the main Tool Shed).
>>
>> This could be some silly mistake in my tar-ball, but usually missing
>> test files and the like get an explicit error. The tests are passing
>> on my GitHub/TravisCI setup (using Twill and the API backend):
>> e.g. https://travis-ci.org/peterjc/galaxy_blast/builds/30592097
>>
>> Here is the current error (the same for the last few Test Tool Shed
>> runs), https://testtoolshed.g2.bx.psu.edu/view/peterjc/blast_rbh
>>
>> Traceback (most recent call last):
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
>> line 108, in test_tool
>>  self.do_it( td )
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
>> line 35, in do_it
>>  self._verify_outputs( testdef, test_history, shed_tool_id,
>> data_list, galaxy_interactor )
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
>> line 69, in _verify_outputs
>>  galaxy_interactor.verify_output( history, output_data,
>> output_testdef=output_testdef, shed_tool_id=shed_tool_id,
>> maxseconds=maxseconds )
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
>> line 53, in verify_output
>>  self.wait_for_history( history_id, maxseconds )
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
>> line 107, in wait_for_history
>>  self.twill_test_case.wait_for( lambda: not self.__history_ready(
>> history_id ), maxseconds=maxseconds)
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py",
>> line 2453, in wait_for
>>  result = func()
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
>> line 107, in 
>>  self.twill_test_case.wait_for( lambda: not self.__history_ready(
>> history_id ), maxseconds=maxseconds)
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
>> line 257, in __history_ready
>>  return self._state_ready( state, error_msg="History in error state."
>> )
>>File
>> "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py",
>> line 316, in _state_ready
>>  raise Exception( error_msg )
>> Exception: History in error state.
>>
>> Is a more detailed log available which might help debug this?
>>
>> Thanks,
>>
>> Peter
>>
>> As an aside, this looks like the Test Tool Shed is still using the
>> Twill backend for the functional tests?
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>>
>
___
Pl