Thanks for the quick responses John and Peter! I feel fortified to carry on.

-Jeltje

On Wed, Jan 7, 2015 at 9:03 PM, Peter Cock <p.j.a.c...@googlemail.com>
wrote:

> On Thu, Jan 8, 2015 at 3:27 AM, Jeltje van Baren
> <jeltje.van.ba...@gmail.com> wrote:
> > Hi,
> >
> > I'm trying to find out what the best way is to see if my functional tests
> > work as expected, and I'm also wondering when these tests are (supposed
> to
> > be) run. Here are the details:
> >
> > Following instructions at
> > https://wiki.galaxyproject.org/Admin/Tools/AddToolTutorial, I created a
> > little toolExample tool under galaxy-dist/tools/myTools. I also created
> test
> > input and output in a test-data directory and added a test to the
> > toolExample.xml file:
> >   <tests>
> >     <test>
> >       <param name="input" value="testinput.fa"/>
> >       <output name="out_file1" file="testoutput.txt"/>
> >     </test>
> >   </tests>
> >
> > And yes, the files are named correctly:
> > tools/myTools/test-data/testinput.fa
> > tools/myTools/test-data/testoutput.txt
>
> If you are manually installing the tool directly under the folder
> galaxy-dist/tools/myTools then you should put your test
> files under galaxy-dist/test-data (this is how all the tools
> used to work back before the ToolShed).
>
> If you want to do the tool installation via a ToolShed,
> things are different (more complicated for the tool
> development side, but it becomes easier for the end
> user to install things).
>
> > I added the new tool to config/tools_conf.xml AND to
> > config/tools_conf.xml.sample.
> > Question #1: Why is a .sample file used by run_functional_tests.sh? I
> > thought with .sample extensions were meant as examples for creating your
> own
> > versions, I'm really surprised that any script would want to access
> these.
>
> Historically (for reasons unclear to me), the tools for testing
> were listed in galaxy-dist/tools_conf.xml.sample however
> that was changed (about a year ago? I forget now) to simply
> use galaxy-dist/tools_conf.xml (the master tool listing as used
> before the ToolShed existed).
>
> You might be using different settings in your main ini file,
> but on our setup it is galaxy-dist/tools_conf.xml and not
> galaxy-dist/config/tools_conf.xml (which seems to be in
> use on your system given the new tool showed up on
> restarting Galaxy):
>
> > I restarted galaxy to verify that the new tool showed up,
> > then checked that I could test it:
> >
> > ./run_functional_tests.sh -list | grep fa_gc_content_1
> > It shows up in the list of tools. So then I tested it:
> >
> > ./run_functional_tests.sh -id fa_gc_content_1
>
> Good - although run_functional_tests.sh is now obsolete
> and just a backward compatible alias for run_tests.sh.
>
> If you read this in some (out of date) documentation,
> please say where so we can update it.
>
> > This gives
> > IOError: [Errno 2] No such file or directory:
> >
> '/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/test-data/testinput.fa'
> >
> > So it looks like at some point the testinput should have been copied to
> > galaxy-dist/test-data but wasn't.
>
> No, as noted above, you have to put the test files there
> by hand (as long as you are deliberately doing all your
> tool installation/development avoiding the ToolShed,
> which is quite a common approach if you have a
> separate Galaxy instance for development).
>
> > Question # 2: Is there some setting that makes this test input (and
> output)
> > get copied to the correct location, or do I have to do this manually?
>
> No, unless you go down the ToolShed route, do this manually.
>
> > I also tried testing by uploading the tool to the testtoolshed (as
> jtest1).
> > I deliberatly put the wrong value in the testoutput.txt file because I'd
> > like to see a test fail before I'll accept that it succeeds. It appears
> the
> > testinput does not get tested on upload, because I got no automated test
> > results, and no error messages anywhere. Is that expected behavior? I
> know
> > the tools get tested every so often, I'm just surprised it doesn't
> happen on
> > upload.
> > The tool could then also be easily installed to my galaxy distribution,
> > without errors in my log or on the browser pages.
>
> The main and Test Tool Shed should now be running the tests
> every 48 hours or so - this service had been unavailable over
> the Christmas/New Year period but should be OK now.
>
> > Question #3: At what point in creating or downloading a repository
> > are these tests run?
>
> Never, assuming I understand your question.
>
> If you mean installing a tool from the ToolShed into a Galaxy
> instance via the Galaxy administrator options, then I think
> after installation any tests should be run automatically.
>
> > And lastly, #4: is either of these the best way to test my tests, or did
> I
> > overlook something?
> >
> > Thank you!
> >
> > -Jeltje
>
> While developing a new tool (or improving one), I personally
> use a development Galaxy server (not the one our users run
> all their real jobs on), and install the tools manually (as above),
> and then test via run_tests.sh -id my_tool_id.
>
> This is complemented by a TravisCI setup, see:
>
> http://blastedbio.blogspot.jp/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>
> If that works, I upload the tool to the TestToolShed, and wait
> for those test results. If that all works and I am ready to make
> a public release, then I upload to the main ToolShed.
>
> I talked about this (in the context of the NCBI BLAST+
> wrappers) at the GCC2014 conference, slides here:
> http://www.slideshare.net/pjacock/galaxy-blast-gcc2014
>
> There is even a video of my talk (and most of the others)
> on the Galaxy website here:
> https://wiki.galaxyproject.org/Events/GCC2014/Abstracts
>
> I hope that helped. There is a work in progress
> "Best Practices" being worked on here:
> https://wiki.galaxyproject.org/Tools/BestPractices
>
> Regards,
>
> Peter
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to