Re: [galaxy-dev] Functional tests results in Invalid HTTP return code 404/500, allowed codes: 200
Hello,
I still haven't figured out why Galaxy's Twill is returning HTTP code
404 when running the functional tests script on my personal Tool Shed. I
just did another test to make sure that it is indeed returning the wrong
HTTP code.
$ python
Python 2.7.3 (default, Sep 26 2013, 20:08:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import twill.commands as tc
>>> tc.go('http://toolshed.galaxy.domain.org/')
==> at http://shed.galaxy.byobu.info/
>>> print tc.browser.get_code()
200
As you can see, now Twill returns HTTP code 200 as expected. This test
was done with python-twill (0.9-3) on Linux. Galaxy also comes with
Twill 0.9.
To test this further, I made some changes to the Python script
responsible for testing the HTTP return code. Here is the diff:
diff -r 6822f41bc9bb
test/install_and_test_tool_shed_repositories/base/twilltestcase.py
---
a/test/install_and_test_tool_shed_repositories/base/twilltestcase.py
Mon Aug 19 13:06:17 2013 -0400
+++
b/test/install_and_test_tool_shed_repositories/base/twilltestcase.py
Sat Oct 19 14:53:26 2013 +0200
@@ -67,7 +67,7 @@
# install_repositories_by_revision redirect, so we have to
include 403 in the allowed HTTP
# status codes and log in again.
url = '%s/repository/install_repositories_by_revision?%s' % (
tool_shed_url, install_params )
-self.visit_url( url, allowed_codes=[ 200, 403 ] )
+self.visit_url( url, allowed_codes=[ 200, 403, 404 ] )
self.logout()
self.login( email='t...@bx.psu.edu', username='test' )
install_params = urllib.urlencode( dict(
repository_ids=encoded_repository_id,
@@ -105,7 +105,7 @@
self.check_for_strings( post_submit_strings_displayed,
strings_not_displayed )
repository_ids = self.initiate_installation_process(
new_tool_panel_section=new_tool_panel_section )
self.wait_for_repository_installation( repository_ids )
-def visit_url( self, url, allowed_codes=[ 200 ] ):
+def visit_url( self, url, allowed_codes=[ 200, 404 ] ):
new_url = tc.go( url )
return_code = tc.browser.get_code()
assert return_code in allowed_codes, 'Invalid HTTP return code
%s, allowed codes: %s' % \
So I made the script accept the 404 return code, to see if it can now
successfully run the functional tests of my tool, and it did!
# 2013-10-19 14:38:58 - repository installation and testing script
completed.
# Repository revisions tested: 1
#
--
# 1 repositories passed all tests:
#
# usearch owned by serrano, changeset revision 095baaf5d877
There are two tests in this repository and it passed both. I think this
proves that Twill indeed returns the wrong HTTP code. It returns 404
(and sometimes 500) when it should return 200. I have no idea why this
happens in Galaxy and I'm hoping that someone else can have a look at it.
Regards,
Serrano
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to make a Tool Shed private?
My bad, I should have been clearer. What I mean is that no one should be able to access it unless I give explicit permission. Browsing or installing tools shouldn't be possible either without permission. Serrano On 09/23/2013 02:54 PM, Greg Von Kuster wrote: > It's a bit unclear what you mean by "private" here, but it's possible > to at least force users to login to use a Tool Shed if you include the > following setting in the [app:main] section your tool_shed_wsgi.ini file. > > # Force everyone to log in (disable anonymous access) > require_login = True > > Greg Von Kuster > > On Sep 22, 2013, at 2:39 PM, Serrano Pereira > mailto:serrano.pere...@gmail.com>> wrote: > >> Hello, >> >> Is it possible to make a Galaxy Tool Shed private? The only method I can >> think of at this moment is to use Apache's configuration to restrict >> access to certain IP addresses. I was wondering if there are alternative >> methods. >> >> Regards, >> Serrano >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] How to make a Tool Shed private?
Hello, Is it possible to make a Galaxy Tool Shed private? The only method I can think of at this moment is to use Apache's configuration to restrict access to certain IP addresses. I was wondering if there are alternative methods. Regards, Serrano ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Functional tests results in Invalid HTTP return code 404/500, allowed codes: 200
Dave, The space isn't there in my actual XML file. I must have accidentally put the space there while constructing the email. The URL that fails with the 404 error is actually correct (as stated in my first post). This means that it successfully obtained the URL from the XML file. So I suspect the problem is somewhere else. Serrano On 09/16/2013 09:30 PM, Dave Bouvier wrote: > It looks like you might have a space in your tool_sheds_conf.xml after > domain.org/ but before the " character, is that the case? If so, I > would suggest removing it and trying again. > >--Dave B. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Functional tests results in Invalid HTTP return code 404/500, allowed codes: 200
Dave, Thanks for your reply. I already had my tool shed specified in tool_sheds_conf.xml, which is why I omitted the GALAXY_INSTALL_TEST_TOOL_SHEDS_CONF environment variable. I just tried it with, GALAXY_INSTALL_TEST_TOOL_SHEDS_CONF=install_and_test_tool_sheds_conf.xml The contents of the install_and_test_tool_sheds_conf.xml is as follows: http://toolshed.galaxy.domain.org/ "/> But it results in the exact same error: AssertionError: Invalid HTTP return code 404, allowed codes: 200 Do you have any other ideas? Regards, Serrano On 09/16/2013 08:37 PM, Dave Bouvier wrote: > Serrano, > > The GALAXY_INSTALL_TEST_TOOL_SHEDS_CONF environment variable should be > set to an XML file that specifies the tool shed you're installing > from. I've updated the wiki page to be clearer on that point. > > --Dave B. > > On 09/16/2013 12:32 PM, Serrano Pereira wrote: >> Dear Galaxy hackers, >> >> I'm having trouble with the tool test framework. I have a local >> installation of a Galaxy Tool Shed (latest stable from the repository) >> on my Linux server. I have an Apache proxy to my tool shed (per >> http://wiki.galaxyproject.org/HostingALocalToolShed). The Tool Shed >> seems to work fine. I'm able to create and maintain repositories and I'm >> able to install tools from the Tool Shed from remote Galaxy instances. >> >> Now I want to run the functional tests of the tools in the Tool Shed. I >> do that by following the instructions on the wiki >> (http://wiki.galaxyproject.org/AutomatedToolTests). It comes down to the >> following commands: >> >> -- >> cd /opt/galaxy/galaxy-dist/ >> export >> GALAXY_INSTALL_TEST_TOOL_SHED_API_KEY=2aeecfe2d9f28d0c5b6959a41886b5fd >> export >> GALAXY_INSTALL_TEST_TOOL_SHED_URL=http://toolshed.galaxy.domain.org/ >> python lib/tool_shed/scripts/check_repositories_for_functional_tests.py >> tool_shed_wsgi.ini >> sh install_and_test_tool_shed_repositories.sh >> -- >> >> The second script (install_and_test_tool_shed_repositories.sh) fails >> with the following error: >> >> == >> FAIL: install_repository_usearch >> (install_and_test_tool_shed_repositories.functional.test_install_repositories.TestInstallRepository_usearch) >> >> Install the repository usearch from http://toolshed.galaxy.domain.org/. >> -- >> Traceback (most recent call last): >>File >> "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py", >> >> line 48, in test_install_repository >> self.do_installation( repository_dict ) >>File >> "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py", >> >> line 17, in do_installation >> self.install_repository( repository_info_dict ) >>File >> "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py", >> >> line 61, in install_repository >> self.visit_url( '%s/repository/preview_tools_in_changeset?%s' % ( >> tool_shed_url, preview_params ) ) >>File >> "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py", >> >> line 112, in visit_url >> ( return_code, url, ', '.join( str( code ) for code in >> allowed_codes ) ) >> AssertionError: Invalid HTTP return code 404 for >> http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12&changeset_revision=095baaf5d877, >> >> allowed codes: 200 >> -- >> >> (I've changed twilltestcase.py to also print the URL that is being >> browsed.) This is were it get's weird. The HTTP 404 response code >> indicates that the page was not found. However, when I browse to that >> URL, it works fine! What's more, when I try to access that page with >> `wget` from the same machine the HTTP response code is 200: >> >> -- >> wget >> http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12&changeset_revision=095baaf5d877 >> >> ... >> HTTP request sent, awaiting response...
[galaxy-dev] Functional tests results in Invalid HTTP return code 404/500, allowed codes: 200
Dear Galaxy hackers, I'm having trouble with the tool test framework. I have a local installation of a Galaxy Tool Shed (latest stable from the repository) on my Linux server. I have an Apache proxy to my tool shed (per http://wiki.galaxyproject.org/HostingALocalToolShed). The Tool Shed seems to work fine. I'm able to create and maintain repositories and I'm able to install tools from the Tool Shed from remote Galaxy instances. Now I want to run the functional tests of the tools in the Tool Shed. I do that by following the instructions on the wiki (http://wiki.galaxyproject.org/AutomatedToolTests). It comes down to the following commands: -- cd /opt/galaxy/galaxy-dist/ export GALAXY_INSTALL_TEST_TOOL_SHED_API_KEY=2aeecfe2d9f28d0c5b6959a41886b5fd export GALAXY_INSTALL_TEST_TOOL_SHED_URL=http://toolshed.galaxy.domain.org/ python lib/tool_shed/scripts/check_repositories_for_functional_tests.py tool_shed_wsgi.ini sh install_and_test_tool_shed_repositories.sh -- The second script (install_and_test_tool_shed_repositories.sh) fails with the following error: == FAIL: install_repository_usearch (install_and_test_tool_shed_repositories.functional.test_install_repositories.TestInstallRepository_usearch) Install the repository usearch from http://toolshed.galaxy.domain.org/. -- Traceback (most recent call last): File "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py", line 48, in test_install_repository self.do_installation( repository_dict ) File "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py", line 17, in do_installation self.install_repository( repository_info_dict ) File "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py", line 61, in install_repository self.visit_url( '%s/repository/preview_tools_in_changeset?%s' % ( tool_shed_url, preview_params ) ) File "/opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py", line 112, in visit_url ( return_code, url, ', '.join( str( code ) for code in allowed_codes ) ) AssertionError: Invalid HTTP return code 404 for http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12&changeset_revision=095baaf5d877, allowed codes: 200 -- (I've changed twilltestcase.py to also print the URL that is being browsed.) This is were it get's weird. The HTTP 404 response code indicates that the page was not found. However, when I browse to that URL, it works fine! What's more, when I try to access that page with `wget` from the same machine the HTTP response code is 200: -- wget http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12&changeset_revision=095baaf5d877 ... HTTP request sent, awaiting response... 200 OK ... -- When I use the Tool Shed URL http://localhost:9009/ instead, I get a similar error, only the return code is 500 instead of 404. I suspect that the HTTP return code returned by the Python module that returns it (Mechanize?) is invalid. I found a note about this error on the wiki (http://wiki.galaxyproject.org/Admin/Running%20Tests): "Note: If you have another version of paste installed in your PYTHONPATH, it can cause problems running the functional tests. Many tests will error (instead of passing/failing) because of an HTML code 500 where a 200 was expected. There are two workarounds for this: uninstall paste or use virtualenv. " The thing is, I do not have another version of Paste installed. What can I do to fix this error? Regards, Serrano Pereira ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
