On Tue, May 7, 2013 at 10:59 AM, Greg Von Kuster <g...@bx.psu.edu> wrote:
> Hi Peter,
>
> This change would be simple to make, but it would result in the inclusion
> of all repositories with missing test components to be included in the list
> of failing tests.  There are currently 2 filters that provide the
> information to categorize these repositories: missing test components ( True
> / False ) and tools functionally correct ( True / False ).  I'm not sure it
> makes sense to add a 3rd filter, but if we can come up with a sensible one,
> I would be happy to add it.  If not, do you want the list of failing tool
> tests to include repositories that are missing test components?

By missing test components do you mean things like missing a test input
or output file? If so, that would be fine. If I've written a test but
not included
the sample data, then to me that is just a special case of a failing test that
needs fixing (and it should be a trivial fix).

Currently (according to the yellow notes on the Test Tool Shed),

Latest revision: missing tool tests:
 * you are authorized to update them
 * the latest installable revision contains at least 1 tool with no
defined tests OR:
 * the latest installable revision contains at least 1 tool with a
test that requires a missing test data file

Latest revision: failing tool tests:
 * you are authorized to update them
 * the latest installable revision contains at least 1 tool
 * the latest installable revision is not missing any tool test components
 * the latest installable revision has at least 1 tool test that fails

My suggestion would be to treat missing test data files as a failing
test, something like this:

Latest revision: missing tool tests:
 * you are authorized to update them
 * the latest installable revision contains at least 1 tool with no
defined tests

Latest revision: failing tool tests:
 * you are authorized to update them
 * the latest installable revision has at least 1 tool test that fails
OR requires a missing test data file

Under this scheme, right now on the Test Tool Shed my ncbi_blast_plus
repository would appear on both lists due to missing tests for tools like
ncbi_rpsblast_wrapper, and failing tests like ncbi_blastn_wrapper (due
to a glitch in the dependency installation changes I made).

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to