On Sun, Nov 17, 2013 at 8:30 AM, John Chilton wrote:
> On Thu, Apr 25, 2013 at 9:36 AM, Peter Cock wrote:
>> ...
>>
>> As noted in my last email, for some reason when running the test case,
>> the input FASTA file is being included on the command line TWICE.
>> Curiously the -hash_index argument has been omitted. Linked maybe?
>
> Peter,
>
> I have fixed the double listing of the FASTA file. Putting min=1 on a
> repeat statement would result in two repeat instances when using
> functional tests without this bug fix.
>
> https://bitbucket.org/galaxy/galaxy-central/commits/5e534cc8da856ad598d63b8b9f03fe20629df05f

Thank you - such a little thing once you'd traced its cause.

> It is likely also the problem with your mira tests?
>

This should help for the MIRA4 tests too :)

> The hash_index
> missing was caused because to the param value you put in the test tag
> should be true or false, not the truevalue/falsevalue attributes as
> far as I can tell - those are used only by cheetah I guess. Adding the
> hash_index parameter creates and additional 5 files - including ones
> you listed in your test case.

I think I tried that too (true/false), but it was a while ago now. Hitting
multiple test framework issues at the same time made debugging
this hard.

> With these change, I was able to write working functional tests for
> your tool using the template you outlined in the Trello card. The .pin
> file doesn't match, I think there is something time-based in there so
> I had to set two lines of diff.

Yes, I agree the PIN file varies run to run, so the diff trick looks good.

> Also, since this e-mail, you now have
> two parameters named file, that doesn't go over well yet - so I
> renamed mask|file to mask|mask_file.

The makeblastdb wrapper on the main Tool Shed don't yet
have the masking file parameter, but does already on the
Test Tool Shed - so I'd prefer not to change this:
http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus
http://testtoolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus

In principle the pipe-based fully specified parameter name
would work here too, to resolve the potential ambiguity?
(That is a separate Trello Card for handling potentially
ambiguous parameters in the test framework):
https://trello.com/c/zSTrfDOB/820-disambiguated-conditional-parameters-not-supported-in-functional-tests

> ...
>
> These changes should work right out of central, does not utilize my
> API driven variant on github.
>
> I discovered no problems with auto_primary versus basic composite
> types here, just the things listed above.

Not for me though, even if I rename the masking "file" param to
avoid the ambiguous "file" parameters, commits here:
https://github.com/peterjc/galaxy_blast/commit/2043cc813c2e138d93f8a940ea711ad7538a50c7
https://github.com/peterjc/galaxy_blast/commit/f4f74cd065921b069499a5fcc4209b299a403c96
and continuing on this branch:
https://github.com/peterjc/galaxy_blast/tree/test_makeblastdb
https://github.com/peterjc/galaxy_blast/commit/e828543539ab0a0ca8f8b16dbdce132812605282

That gave:

$ ./run_functional_tests.sh -id ncbi_makeblastdb
...
galaxy.jobs.runners.local DEBUG 2013-11-18 11:25:38,449 execution
finished: export GALAXY_SLOTS="1"; makeblastdb -version &>
/tmp/tmpF81TF5/tmpghF4ik/new_files_path_lVhJ4E/GALAXY_VERSION_STRING_2;
makeblastdb -out
"/tmp/tmpF81TF5/tmpghF4ik/database/files/000/dataset_2_files/blastdb"
-hash_index -in "
/tmp/tmpF81TF5/tmpghF4ik/database/files/000/dataset_1.dat
/tmp/tmpF81TF5/tmpghF4ik/database/files/000/dataset_1.dat " -title
"Just 4 human proteins" -dbtype prot
...
Composite file (blastdb.phr) of History item 2 different than
expected, difference (using diff):
Binary data detected, not displaying diff
...
FAILED (failures=1)
...

I find that changing the order of the <extra_files> tags in the test seems
to alter the failure - which supports my hunch that something is
scrambling the order of the extra files, so that it fails to compare the
generated blastdb.phr with the provided four_human_proteins.fasta.phd

e.g. Here it seems to compare to the (place holder) text I generate
when viewing a database in the Galaxy interface:
https://github.com/peterjc/galaxy_blast/commit/e828543539ab0a0ca8f8b16dbdce132812605282

$ ./run_functional_tests.sh -id ncbi_makeblastdb
...
galaxy.jobs.runners.local DEBUG 2013-11-18 12:23:24,812 execution
finished: export GALAXY_SLOTS="1"; python
/mnt/galaxy/galaxy-central/tools/data_source/upload.py
/mnt/galaxy/galaxy-central
/tmp/tmph0YlBO/tmpRK7bhO/new_files_path_0SO6J0/tmpkPWlQd
/tmp/tmph0YlBO/tmpRK7bhO/new_files_path_0SO6J0/tmpr_xBCK
1:/tmp/tmph0YlBO/tmpRK7bhO/job_working_directory_Qh_YcY/000/1/dataset_1_files:/tmp/tmph0YlBO/tmpRK7bhO/database/files/000/dataset_1.dat
...
Composite file (blastdb.phd) of History item 2 different than
expected, difference (using diff):
--- local_file
+++ history_data
@@ -1,4 +1,1 @@
-11117184492
-29249033410
-36665887501
-5392473183
+This is a BLAST protein database.
...
FAILED (failures=1)
...

If it works for you then perhaps the filesystem is a factor, e.g.
os.listdir(...) order?
I have had an initial look at the code in test/base/twilltestcase.py but haven't
spotted a problem yet.

Thank you John,

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to