Hi Peter,

On May 7, 2013, at 9:10 AM, Peter Cock wrote:

> On Tue, May 7, 2013 at 10:59 AM, Greg Von Kuster <g...@bx.psu.edu> wrote:
>> Hi Peter,
>> 
>> This change would be simple to make, but it would result in the inclusion
>> of all repositories with missing test components to be included in the list
>> of failing tests.  There are currently 2 filters that provide the
>> information to categorize these repositories: missing test components ( True
>> / False ) and tools functionally correct ( True / False ).  I'm not sure it
>> makes sense to add a 3rd filter, but if we can come up with a sensible one,
>> I would be happy to add it.  If not, do you want the list of failing tool
>> tests to include repositories that are missing test components?
> 
> By missing test components do you mean things like missing a test input
> or output file? If so, that would be fine. If I've written a test but
> not included
> the sample data, then to me that is just a special case of a failing test that
> needs fixing (and it should be a trivial fix).


Missing test components implies a tool config that does not define a test (i.e, 
a missing test definition) or a tool config that defines a test, but the test's 
input or output files are missing from the repository.


> 
> Currently (according to the yellow notes on the Test Tool Shed),
> 
> Latest revision: missing tool tests:
> * you are authorized to update them
> * the latest installable revision contains at least 1 tool with no
> defined tests OR:
> * the latest installable revision contains at least 1 tool with a
> test that requires a missing test data file
> 
> Latest revision: failing tool tests:
> * you are authorized to update them
> * the latest installable revision contains at least 1 tool
> * the latest installable revision is not missing any tool test components
> * the latest installable revision has at least 1 tool test that fails
> 
> My suggestion would be to treat missing test data files as a failing
> test, something like this:
> 
> Latest revision: missing tool tests:
> * you are authorized to update them
> * the latest installable revision contains at least 1 tool with no
> defined tests

I don't see the benefit of the above where you place tools missing tests into a 
different category than tools with defined tests, but missing test data.  If 
any of the test components (test definition or required input or output files) 
are missing, then the test cannot be executed, so defining it as a failing test 
in either case is a bit misleading.  It is actually a tool that is missing test 
components that are required for execution which will result in a pass / fail 
status.

It would be much simpler to change the filter for failing tests to include 
those that are missing test components so that the list of missing test 
components is a subset of the list of failing tests.

> 
> Latest revision: failing tool tests:
> * you are authorized to update them
> * the latest installable revision has at least 1 tool test that fails
> OR requires a missing test data file
> 
> Under this scheme, right now on the Test Tool Shed my ncbi_blast_plus
> repository would appear on both lists due to missing tests for tools like
> ncbi_rpsblast_wrapper, and failing tests like ncbi_blastn_wrapper (due
> to a glitch in the dependency installation changes I made).
> 
> Peter


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to