[galaxy-dev] errors with run_functional_tests.sh

2013-03-05 Thread Florence Maurier

Hi,

We are currently installing galaxy on a server. By reading the galaxy 
wiki, we found the script run_functional_tests.sh (the principle is 
really handy!). Unfortunately, we encounter errors when we use it. I 
hope someone can help us.


In reality, we encounter several types of errors ...
The first is present in each run, for example for the execution follows:

./run_functional_tests.sh -id fastqc
...
FastQC:Read QC ( fastqc )  Test-1 ... galaxy.web.framework DEBUG 
2013-03-01 09:42:10,466

...
galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:13,357 tool 
upload1 created job id 1

...
galaxy.jobs DEBUG 2013-03-01 09:42:27,436 Tool did not define exit code 
or stdio handling; checking stderr for success

galaxy.jobs DEBUG 2013-03-01 09:42:27,757 job 1 ended

galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:40,883 tool 
upload1 created job id 2

...
galaxy.jobs DEBUG 2013-03-01 09:42:52,000 Tool did not define exit code 
or stdio handling; checking stderr for success

galaxy.jobs DEBUG 2013-03-01 09:42:52,311 job 2 ended

galaxy.jobs.manager DEBUG 2013-03-01 09:42:55,285 (3) Job assigned to 
handler 'main'
galaxy.jobs DEBUG 2013-03-01 09:43:00,477 (3) Working directory for job 
is: /var/local/galaxy/galaxy-dist/database/job_working_directory/000/3
galaxy.jobs.handler DEBUG 2013-03-01 09:43:00,478 dispatching job 3 to 
local runner

galaxy.jobs.handler INFO 2013-03-01 09:43:00,622 (3) Job dispatched
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:00,727 Local runner: 
starting job 3
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:01,193 executing: 
python /var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i 
/tmp/tmp_YcL4J/database/files/000/dataset_1.dat -d 
/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files 
-o /tmp/tmp_YcL4J/database/files/000/dataset_3.dat -n fastqcout -f 
fastq -j 1000gsample.fastq -e 
/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc -c 
/tmp/tmp_YcL4J/database/files/000/dataset_2.dat
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:04,515 execution 
finished: python 
/var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i 
/tmp/tmp_YcL4J/database/files/000/dataset_1.dat -d 
/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files 
-o /tmp/tmp_YcL4J/database/files/000/dataset_3.dat -n fastqcout -f 
fastq -j 1000gsample.fastq -e 
/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc -c 
/tmp/tmp_YcL4J/database/files/000/dataset_2.dat
galaxy.jobs DEBUG 2013-03-01 09:43:04,578 Tool did not define exit code 
or stdio handling; checking stderr for success
_galaxy.tools DEBUG 2013-03-01 09:43:04,814 Error opening galaxy.json 
file: [Errno 2] No such file or directory: 
'/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/galaxy.json'_

galaxy.jobs DEBUG 2013-03-01 09:43:04,899 job 3 ended
ok
--
Ran 1 test in 60.889s
OK
...
Exception OSError: (2, 'No such file or directory', 
'/tmp/tmp_YcL4J/database/tmp/tmpn5w0Ln') in bound method 
_TemporaryFileWrapper.__del__ of closed file 'fdopen', mode 'w' at 
0x18ffd5d0 ignored



According to other messages from the mailing list, my understanding is 
that this message should be ignored and we only have to consider the 
OK that says the test went well. Is that correct ?


The other type of error we find is the following:
./run_functional_tests.sh -id cshl_fastx_clipper
...
ERROR: Failure: ValueError (No such test TestForTool_cshl_fastx_clipper)
--
Traceback (most recent call last):
  File 
/var/local/galaxy/galaxy-dist/eggs/nose-0.11.1-py2.6.egg/nose/failure.py, 
line 39, in runTest

raise self.exc_class(self.exc_val)
ValueError: No such test TestForTool_cshl_fastx_clipper
--
...
For some of these tools, the error seems logical given that the xml 
configuration tool does not contain test. However, for others like 
fastx_clipper, the test is described... So, why is there a problem with 
this kind of tools ?


Finally, we have one last type of error, like this:
./run_functional_tests.sh -id sam_to_bam
...
SAM-to-BAM ( sam_to_bam )  Test-2 ... galaxy.web.framework DEBUG 
2013-03-05 18:02:15,115 Error: this request returned None from 
get_history(): http://localhost:9051/user/logout
galaxy.web.framework DEBUG 2013-03-05 18:02:15,159 Error: this request 
returned None from get_history(): http://localhost:9051/

ERROR

==
ERROR: SAM-to-BAM ( sam_to_bam )  Test-2
--
Traceback (most recent call last):
  File /var/local/galaxy/galaxy-dist/test/functional/test_toolbox.py, 
line 171, in test_tool

self.do_it( td, shed_tool_id=shed_tool_id )
  File 

Re: [galaxy-dev] errors with run_functional_tests.sh

2013-03-05 Thread Dave Bouvier

Hello,

I'll respond inline, but your base understanding is correct.

   --Dave B.

On 3/5/13 12:17:47.000, Florence Maurier wrote:

Hi,

We are currently installing galaxy on a server. By reading the galaxy
wiki, we found the script run_functional_tests.sh (the principle is
really handy!). Unfortunately, we encounter errors when we use it. I
hope someone can help us.

In reality, we encounter several types of errors ...
The first is present in each run, for example for the execution follows:

./run_functional_tests.sh -id fastqc
...
FastQC:Read QC ( fastqc )  Test-1 ... galaxy.web.framework DEBUG
2013-03-01 09:42:10,466
...
galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:13,357 tool
upload1 created job id 1
...
galaxy.jobs DEBUG 2013-03-01 09:42:27,436 Tool did not define exit code
or stdio handling; checking stderr for success
galaxy.jobs DEBUG 2013-03-01 09:42:27,757 job 1 ended

galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:40,883 tool
upload1 created job id 2
...
galaxy.jobs DEBUG 2013-03-01 09:42:52,000 Tool did not define exit code
or stdio handling; checking stderr for success
galaxy.jobs DEBUG 2013-03-01 09:42:52,311 job 2 ended

galaxy.jobs.manager DEBUG 2013-03-01 09:42:55,285 (3) Job assigned to
handler 'main'
galaxy.jobs DEBUG 2013-03-01 09:43:00,477 (3) Working directory for job
is: /var/local/galaxy/galaxy-dist/database/job_working_directory/000/3
galaxy.jobs.handler DEBUG 2013-03-01 09:43:00,478 dispatching job 3 to
local runner
galaxy.jobs.handler INFO 2013-03-01 09:43:00,622 (3) Job dispatched
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:00,727 Local runner:
starting job 3
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:01,193 executing:
python /var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i
/tmp/tmp_YcL4J/database/files/000/dataset_1.dat -d
/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files
-o /tmp/tmp_YcL4J/database/files/000/dataset_3.dat -n fastqcout -f
fastq -j 1000gsample.fastq -e
/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc -c
/tmp/tmp_YcL4J/database/files/000/dataset_2.dat
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:04,515 execution
finished: python
/var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i
/tmp/tmp_YcL4J/database/files/000/dataset_1.dat -d
/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files
-o /tmp/tmp_YcL4J/database/files/000/dataset_3.dat -n fastqcout -f
fastq -j 1000gsample.fastq -e
/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc -c
/tmp/tmp_YcL4J/database/files/000/dataset_2.dat
galaxy.jobs DEBUG 2013-03-01 09:43:04,578 Tool did not define exit code
or stdio handling; checking stderr for success
_galaxy.tools DEBUG 2013-03-01 09:43:04,814 Error opening galaxy.json
file: [Errno 2] No such file or directory:
'/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/galaxy.json'_

galaxy.jobs DEBUG 2013-03-01 09:43:04,899 job 3 ended
ok
--
Ran 1 test in 60.889s
OK
...
Exception OSError: (2, 'No such file or directory',
'/tmp/tmp_YcL4J/database/tmp/tmpn5w0Ln') in bound method
_TemporaryFileWrapper.__del__ of closed file 'fdopen', mode 'w' at
0x18ffd5d0 ignored


According to other messages from the mailing list, my understanding is
that this message should be ignored and we only have to consider the
OK that says the test went well. Is that correct ?


Yes, the error messages you see there are irrelevant, and can be safely 
ignored. As you deduced, the functional test passed.




The other type of error we find is the following:
./run_functional_tests.sh -id cshl_fastx_clipper
...
ERROR: Failure: ValueError (No such test TestForTool_cshl_fastx_clipper)
--
Traceback (most recent call last):
   File
/var/local/galaxy/galaxy-dist/eggs/nose-0.11.1-py2.6.egg/nose/failure.py,
line 39, in runTest
 raise self.exc_class(self.exc_val)
ValueError: No such test TestForTool_cshl_fastx_clipper
--
...
For some of these tools, the error seems logical given that the xml
configuration tool does not contain test. However, for others like
fastx_clipper, the test is described... So, why is there a problem with
this kind of tools ?


In the case of fastx_clipper, the test definition is commented out, and 
therefore not found in the toolbox.




Finally, we have one last type of error, like this:
./run_functional_tests.sh -id sam_to_bam
...
SAM-to-BAM ( sam_to_bam )  Test-2 ... galaxy.web.framework DEBUG
2013-03-05 18:02:15,115 Error: this request returned None from
get_history(): http://localhost:9051/user/logout
galaxy.web.framework DEBUG 2013-03-05 18:02:15,159 Error: this request
returned None from get_history(): http://localhost:9051/
ERROR

==
ERROR: