Re: [galaxy-dev] tools installed from a toolshed repository displayed in an arbitrary order
Hi Yulia, that is currently not supported. I think there is somewhere a trello card about that problem. One idea was to give each repository a hint where to install the tools (maybe be a small tool_conf.xml). Maybe you have a better idea? Currently you can try to manipulate the integrated_tool_panel.xml file, but that file can be overwritten at certain points. It is regenerated from all of your used tool_conf files. Hope that helps, Bjoern Hello guys, After porting our Galaxy tools into a tool shed repository the tool sections got lost. I know there is a possibility to assign a repository to a section inside the tools panel when the repository is getting installed. However, we would still like to have sections inside the repository – that makes navigating the tools much neater. Also, the way shed_tool_conf.xml is populated with tools during repository installation seems arbitrary to me. Is there any way to control this, to have repository tools sorted/displayed in a certain order in Tools panel, in the very least, alphabetically? I’ve got many tools in my repository (and no, it is not possible to divide them into several repositories), and the way they are sorted or divided into sections makes navigating the tools much easier for a user. Thanks, Yulia ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Resolved toolshed mercurial authentication issue, public name change issue remains
Am Sonntag, den 15.09.2013, 22:01 + schrieb Guest, Simon:
Hi Greg,
I finally got to the bottom of the reasons for my abort: authorization failed
message when trying to push tool updates into my local toolshed using hg
command line (rather than web interface), having given up for a while, and
just come back to it.
Since my authentication agent (apache LDAP module using corporate Active
Directory) was providing bare usernames to the Toolshed, I had been relying
on the remote_user_maildomain feature to turn this into an email address.
However, it seems this function is not triggered when using hg command line
access, so I had to change my rewrite rule which sets up the apache
REMOTE_USER in my apache conf file for the toolshed, from this:
RequestHeader set REMOTE_USER %{RU}e
to this:
RequestHeader set REMOTE_USER %{RU}e...@agresearch.co.nz
Hi Simon,
thanks Simon.
Bjoern
Now I can push updates directly from hg, and it's much more convenient. I
didn't have to change anything in that toolshed middleware (although some
temporary log messages I added there were the key for me solving the problem).
However, that bug remains where I can continue to change my public name in
the web user interface, even after I have created repos. I know not to do
this. If there's more information you want from me to try to track down what
is going wrong here, just say what you need, and I will endeavour to provide
it. But for me, this is now a minor issue, so don't do anything just on my
account.
Thanks for your help.
cheers,
Simon
-Original Message-
From: Greg Von Kuster [mailto:g...@bx.psu.edu]
Sent: Friday, 6 September 2013 1:00 p.m.
To: Guest, Simon
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks
repo path
Hello Simon,
The tool shed middleware basically consists of 2 files in
~/lib/galaxy/webapps/tool_shed/framework/middleware: hg.py which handles
authentication when pushing from the command line. The remoteuser.py file
handles remote user authentication, but I cannot guarantee it properly
handles hg push from the command line since I don't have an environment
set up that way. That would be the place to take a look though, and if
you discover problems let me know and we'll apply a patch.
Thanks very much,
Greg Von Kuster
On Sep 5, 2013, at 4:53 PM, Guest, Simon simon.gu...@agresearch.co.nz
wrote:
Hi Greg,
I just changed the public name on the web interface, where you click on
the User dropdown, and choose Public Name. Something about my environment
let it happen, when it apparently should have not been possible.
I suspect the reason lies in some mismatch between my apache user
authentication and toolshed userids. See my previous email for the
details of what I've been doing with that. I don't understand the
connection between userids (email addresses) used for authentication, and
public names, which appear to be used by the mercurial server inside the
toolshed (by inference from the public name appearing in the allow_push in
the toolshed repo). Is there a layer of toolshed middleware that maps
between the two?
I am going to investigate other ways of doing the authentication, as I
don't think passing in REMOTE_USER from apache is working (for me). I
would like to know how others are doing that.
cheers,
Simon
-Original Message-
From: Greg Von Kuster [mailto:g...@bx.psu.edu]
Sent: Friday, 6 September 2013 1:17 a.m.
To: Guest, Simon
Cc: Dave Bouvier; galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Bug in toolshed: changing public name
breaks repo path
Hello Simon,
On Sep 4, 2013, at 5:13 PM, Guest, Simon
simon.gu...@agresearch.co.nz
wrote:
Hi Dave, Greg,
Thanks for your reply.
I'm running a recently checked out stable branch. hg log shows this
tip:
changeset: 10473:c42567f43aa7
tag: tip
user:greg
date:Mon Aug 19 13:19:56 2013 -0400
summary: Filter invalid objects when generating the list of
repository_dependencies objects that are associated with a tool shed
repository installed into Galaxy.
I wonder if this problem is related to my more fundamental problem
of
being unable to get mercurial push authentication working properly.
I care much more about that, as that's blocking me (whereas the
problem I reported has an easy work-around).
Even though you've found a work-around to the problem that results
when you change your public user name in the tool shed after you have
created a repository that uses it, it is extremely important that we
get a fix ffor the issue committed to the tool shed code. Can you
let us know how you went about changing your public user name? The
only way we've been able to do it is
Re: [galaxy-dev] tools installed from a toolshed repository displayed in an arbitrary order
IIRC then you can change the order of tools manually by copying around the individual tools in shed_tool_conf.xml In my case it builds the integrated_tool_panel.xml in reverse order within the section. I still have labels between the section defined in the tool_conf.xml. But no subsections. best, ido On Sep 16, 2013, at 6:40 AM, yulia.arzha...@csiro.au wrote: Hello guys, After porting our Galaxy tools into a tool shed repository the tool sections got lost. I know there is a possibility to assign a repository to a section inside the tools panel when the repository is getting installed. However, we would still like to have sections inside the repository – that makes navigating the tools much neater. Also, the way shed_tool_conf.xml is populated with tools during repository installation seems arbitrary to me. Is there any way to control this, to have repository tools sorted/displayed in a certain order in Tools panel, in the very least, alphabetically? I’ve got many tools in my repository (and no, it is not possible to divide them into several repositories), and the way they are sorted or divided into sections makes navigating the tools much easier for a user. Thanks, Yulia ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Running functional tests is too slow
Hi all, I'd like to echo Florent's concern that run_functional_tests.sh is too slow, and that this discourages Tool Authors from adding more tests to their tools: https://trello.com/c/wL21d2do/1017-functional-tests-take-too-long (And encourage people to vote up that issue ;) ) Florent has identified one clear bottleneck is the creation of a fresh SQLite database each time, upon which the growing number of schema migration calls must be performed. Could we not cache an empty but up to date copy of this database? i.e. Something like this: 1. run_functional_tests.sh starts 2. If it exists, copy empty_test_sqlite_database.db, otherwise create new test SQLite database as now. 3. Check the schema version. 4. If the temporary SQlite database is out of date, run the migration, and then save a copy of this now up to date database as empty_test_sqlite_database.db, 5. run the tests Perhaps the empty SQLite database could even be cached in BitBucket too, for a faster first test run with a clean checkout? Separately, running the tests themselves seems overly slow - can anything be tweaked here? For example, is there any point executing the external set_meta script in the test environment? Regards, Peter P.S. On a related note, my achievement last Friday was to get TravisCI doing continuous integration testing of a GitHub repository of Galaxy tools: https://travis-ci.org/peterjc/galaxy_blast https://github.com/peterjc/galaxy_blast For those not familiar with this system, the idea is that via a special .travis.yml configuration file, each time new commits are pushed to GitHub, the latest code is checked out and tested on a virtual machine. i.e. Continuous in the sense of running the tests after each change, rather than just once a night. Right now the tests for the BLAST+ suite and associated tool wrappers like Blast2GO takes from 15 to 20 minutes, which I feel could be much improved. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running functional tests is too slow
Peter, Yes, the functional test suite is a bit on the slow side, and one of my long-term goals has been to improve the performance as best I can. --Dave B. On 09/16/2013 07:24 AM, Peter Cock wrote: Hi all, I'd like to echo Florent's concern that run_functional_tests.sh is too slow, and that this discourages Tool Authors from adding more tests to their tools: https://trello.com/c/wL21d2do/1017-functional-tests-take-too-long (And encourage people to vote up that issue ;) ) Florent has identified one clear bottleneck is the creation of a fresh SQLite database each time, upon which the growing number of schema migration calls must be performed. Could we not cache an empty but up to date copy of this database? i.e. Something like this: 1. run_functional_tests.sh starts 2. If it exists, copy empty_test_sqlite_database.db, otherwise create new test SQLite database as now. 3. Check the schema version. 4. If the temporary SQlite database is out of date, run the migration, and then save a copy of this now up to date database as empty_test_sqlite_database.db, 5. run the tests And that would be very easy to automate, with the environment variable GALAXY_TEST_DBURI set to the path to the empty, migrated database, and a few aliases or scripts to replace the test database file with the template file on each run. Perhaps the empty SQLite database could even be cached in BitBucket too, for a faster first test run with a clean checkout? I'm not sure of that idea, it would feel a bit like adding a non-essential file to the codebase which is generated programmatically by existing code. Separately, running the tests themselves seems overly slow - can anything be tweaked here? For example, is there any point executing the external set_meta script in the test environment? The switch to the external set_meta script was done because setting metadata internally was becoming computationally expensive. As I understand it, switching to internal metadata would actually lead to poorer performance in the functional tests. Regards, Peter P.S. On a related note, my achievement last Friday was to get TravisCI doing continuous integration testing of a GitHub repository of Galaxy tools: https://travis-ci.org/peterjc/galaxy_blast https://github.com/peterjc/galaxy_blast For those not familiar with this system, the idea is that via a special .travis.yml configuration file, each time new commits are pushed to GitHub, the latest code is checked out and tested on a virtual machine. i.e. Continuous in the sense of running the tests after each change, rather than just once a night. Right now the tests for the BLAST+ suite and associated tool wrappers like Blast2GO takes from 15 to 20 minutes, which I feel could be much improved. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running functional tests is too slow
On Mon, Sep 16, 2013 at 2:30 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, Yes, the functional test suite is a bit on the slow side, and one of my long-term goals has been to improve the performance as best I can. Great - I appreciate there are lots of other high priority issues with the test framework (e.g. repeats and conditionals), so speed alone won't be top of the list. Florent has identified one clear bottleneck is the creation of a fresh SQLite database each time, upon which the growing number of schema migration calls must be performed. Could we not cache an empty but up to date copy of this database? i.e. Something like this: 1. run_functional_tests.sh starts 2. If it exists, copy empty_test_sqlite_database.db, otherwise create new test SQLite database as now. 3. Check the schema version. 4. If the temporary SQlite database is out of date, run the migration, and then save a copy of this now up to date database as empty_test_sqlite_database.db, 5. run the tests And that would be very easy to automate, with the environment variable GALAXY_TEST_DBURI set to the path to the empty, migrated database, and a few aliases or scripts to replace the test database file with the template file on each run. I see that environment variable already exists, perhaps I can hack something together... at least enough to show a worthwhile time saving. Perhaps the empty SQLite database could even be cached in BitBucket too, for a faster first test run with a clean checkout? I'm not sure of that idea, it would feel a bit like adding a non-essential file to the codebase which is generated programmatically by existing code. External caching would be enough (and cleaner). Separately, running the tests themselves seems overly slow - can anything be tweaked here? For example, is there any point executing the external set_meta script in the test environment? The switch to the external set_meta script was done because setting metadata internally was becoming computationally expensive. As I understand it, switching to internal metadata would actually lead to poorer performance in the functional tests. Sorry, I was unclear - although that is interesting. What I meant was, do we even need the metadata for validating the test results? If not, skip generating it. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running functional tests is too slow
--Dave B. On 09/16/2013 09:38 AM, Peter Cock wrote: Separately, running the tests themselves seems overly slow - can anything be tweaked here? For example, is there any point executing the external set_meta script in the test environment? The switch to the external set_meta script was done because setting metadata internally was becoming computationally expensive. As I understand it, switching to internal metadata would actually lead to poorer performance in the functional tests. Sorry, I was unclear - although that is interesting. What I meant was, do we even need the metadata for validating the test results? If not, skip generating it. Unfortunately, the metadata needs to be set in order to populate and validate input fields, and I don't believe it's currently possible to limit metadata setting to uploaded files, and not output datasets. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running functional tests is too slow
On Mon, Sep 16, 2013 at 2:56 PM, Dave Bouvier d...@bx.psu.edu wrote: On 09/16/2013 09:38 AM, Peter Cock wrote: What I meant was, do we even need the metadata for validating the test results? If not, skip generating it. Unfortunately, the metadata needs to be set in order to populate and validate input fields, ... That makes sense, although I personally would think the current approach via the upload tool is an unnecessary overhead in itself - why not work with the input files in the test-data folder directly without copying them? This could also solve the problem of only being able to write tests with input files which support uploading - right now for example I can't write tests using BLAST databases (composite datatypes) as inputs. and I don't believe it's currently possible to limit metadata setting to uploaded files, and not output datasets. Do you think it would save much time if that was disabled? Or is the metadata not that slow and just produces lots of logging output drawing attention to itself? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] CILogon for Galaxy
Hi, I have developed a thin front end client in PHP to enable CILogon authentication for Galaxy, and set up a dummy Galaxy instance demo at https://gw64.iu.xsede.org. Please let me know if this is of interest to anyone and I'd be happy share more details as well as the source. The choice of PHP instead of Python is because I was unable to find a good Python Oauth library that supports the RSA-SHA1 signature method required by CILogon's OAuth interface. Thanks Marie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] UnboundLocalError: local variable 'prior_installation_required' referenced before assignment
Hi, I got this error when trying to install a repository I created defining a tool dependency. The odd thing is the exact same definition worked for the previous version of the same tool. This is the repository failing to install, see full trace below: http://testtoolshed.g2.bx.psu.edu/view/cjav/package_ngs_tools_0_1_5 This is the content of the only file in the repository: ?xml version=1.0? tool_dependency package name=biopython version=1.61 repository name=package_biopython_1_61 owner=biopython prior_installation_required=True / /package package name=ngs-tools version=0.1.5 install version=1.0 actions action type=setup_virtualenvngs-tools==0.1.5/action /actions /install /package /tool_dependency This is the repository for the previous version of this package currently in the test toolshed and can be installed without issues: http://testtoolshed.g2.bx.psu.edu/view/cjav/package_ngs_tools_0_1_4 Diff between tool_dependencies.xml files: $ diff package_ngs_tools_0_1_4/tool_dependencies.xml package_ngs_tools_0_1_5/tool_dependencies.xml 6c6 package name=ngs-tools version=0.1.4 --- package name=ngs-tools version=0.1.5 9c9 action type=setup_virtualenvngs-tools==0.1.4/action --- action type=setup_virtualenvngs-tools==0.1.5/action Full trace: URL: http://localhost:8080/admin_toolshed/prepare_for_install?tool_shed_url=http://testtoolshed.g2.bx.psu.edu/repository_ids=e9d772a27c450e74changeset_revisions=df1454bc4a5e File '/home/cborroto/src/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/cborroto/src/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/cborroto/src/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/cborroto/src/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/home/cborroto/src/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/home/cborroto/src/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 221 in decorator return func( self, trans, *args, **kwargs ) File '/home/cborroto/src/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 911 in prepare_for_install includes_tool_dependencies ) File '/home/cborroto/src/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 85 in get_dependencies_for_repository installed_rd, missing_rd = get_installed_and_missing_repository_dependencies_for_new_install( trans, repo_info_tuple ) File '/home/cborroto/src/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 214 in get_installed_and_missing_repository_dependencies_for_new_install tool_shed, name, owner, changeset_revision, prior_installation_required = suc.parse_repository_dependency_tuple( rd_tup ) File '/home/cborroto/src/galaxy-dist/lib/tool_shed/util/shed_util_common.py', line 1183 in parse_repository_dependency_tuple prior_installation_required = str( prior_installation_required ) UnboundLocalError: local variable 'prior_installation_required' referenced before assignment I clearly have to be missing something here. Any help? Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] UnboundLocalError: local variable 'prior_installation_required' referenced before assignment
Carlos, The issue you're experiencing is due to features being added in the test tool shed that have not yet been accounted for in Galaxy's stable release to galaxy-dist. To resolve this issue, you'll need to be running a recently updated checkout of galaxy-central on the default branch, since the recently introduced features were added and are being handled in that branch. --Dave B. On 09/16/2013 03:57 PM, Carlos Borroto wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos On Mon, Sep 16, 2013 at 3:21 PM, Carlos Borroto carlos.borr...@gmail.com wrote: Hi, I got this error when trying to install a repository I created defining a tool dependency. The odd thing is the exact same definition worked for the previous version of the same tool. This is the repository failing to install, see full trace below: http://testtoolshed.g2.bx.psu.edu/view/cjav/package_ngs_tools_0_1_5 This is the content of the only file in the repository: ?xml version=1.0? tool_dependency package name=biopython version=1.61 repository name=package_biopython_1_61 owner=biopython prior_installation_required=True / /package package name=ngs-tools version=0.1.5 install version=1.0 actions action type=setup_virtualenvngs-tools==0.1.5/action /actions /install /package /tool_dependency This is the repository for the previous version of this package currently in the test toolshed and can be installed without issues: http://testtoolshed.g2.bx.psu.edu/view/cjav/package_ngs_tools_0_1_4 Diff between tool_dependencies.xml files: $ diff package_ngs_tools_0_1_4/tool_dependencies.xml package_ngs_tools_0_1_5/tool_dependencies.xml 6c6 package name=ngs-tools version=0.1.4 --- package name=ngs-tools version=0.1.5 9c9 action type=setup_virtualenvngs-tools==0.1.4/action --- action type=setup_virtualenvngs-tools==0.1.5/action Full trace: URL: http://localhost:8080/admin_toolshed/prepare_for_install?tool_shed_url=http://testtoolshed.g2.bx.psu.edu/repository_ids=e9d772a27c450e74changeset_revisions=df1454bc4a5e File '/home/cborroto/src/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/cborroto/src/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/cborroto/src/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/cborroto/src/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/home/cborroto/src/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/home/cborroto/src/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 221 in decorator return func( self, trans, *args, **kwargs ) File '/home/cborroto/src/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 911 in prepare_for_install includes_tool_dependencies ) File '/home/cborroto/src/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 85 in get_dependencies_for_repository installed_rd, missing_rd = get_installed_and_missing_repository_dependencies_for_new_install( trans, repo_info_tuple ) File '/home/cborroto/src/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 214 in get_installed_and_missing_repository_dependencies_for_new_install tool_shed, name, owner, changeset_revision, prior_installation_required = suc.parse_repository_dependency_tuple( rd_tup ) File '/home/cborroto/src/galaxy-dist/lib/tool_shed/util/shed_util_common.py', line 1183 in parse_repository_dependency_tuple prior_installation_required = str( prior_installation_required ) UnboundLocalError: local variable 'prior_installation_required' referenced before assignment I clearly have to be missing something here. Any help? Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at:
Re: [galaxy-dev] UnboundLocalError: local variable 'prior_installation_required' referenced before assignment
On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] UnboundLocalError: local variable 'prior_installation_required' referenced before assignment
On Mon, Sep 16, 2013 at 4:46 PM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Peter, When I install my repository which has package_biopython_1_61 as a dependency, package_atlas_3_10 and package_lapack_3_4 show as Installed, missing tool dependencies(Grey). However package_biopython_1_61 shows as Installed(Green) and as far as I can tell everything is working with the tools depending on my repository. actually that is correct if you did not manually deactivate CPU throttling. Can I kindly ask which OS do you use. On Ubuntu it can be deactivated automatically, on Fedora you need to be root. Sure, this is on Ubuntu 13.04. I would have to double check, but I believe I saw the same behavior on Centos 6.4. Best, --Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CuffDiff read group tracking
You can overwrite your copy of the Cuffdiff wrapper at tools/ngs_rna/cuffdiff_wrapper.xml with the new version: https://bitbucket.org/galaxy/galaxy-central/raw/48fad71361b3075c85c7d365dd52c106cabad73a/tools/ngs_rna/cuffdiff_wrapper.xml Best, J. On Sep 16, 2013, at 5:36 PM, Matthew Paul wrote: Hey Galaxy, I am trying to use CuffDIff with replicate analysis on a local instance. I saw that Jeremy Goecks had a solution to this problem: http://user.list.galaxyproject.org/Cuffdiff-changes-tt4655893.html#a4655901. Could I see/use this source code please? I can't really wait until the next update. Thanks, Matt Paul ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] workflow issues in local Galaxy....
I am not using CloudMan, but we do use a sqlite database. It seems to have that behavior when there is an output that connects to two inputs... but I think I need to do more investigating. I suppose we could try postgres and see if that works... - Nik. On Mon, Sep 16, 2013 at 2:04 PM, Björn Grüning bjo...@gruenings.eu wrote: Hi Nikhil, do you use the CloudMan?http://wiki.galaxyproject.org/CloudMan I have such a behaviour when I used the sqlite database and not postgresql. Cheers, Bjoern Hi all, So we use Galaxy to teach our bioinformatics courses and we have an install of Galaxy in the Amazon cloud that we have customized with various tools. However, we don't touch the Galaxy code base itself. Recently, we've been getting errors in creating and running workflows. We are getting locked database errors, sometimes the steps seem to run out of order, and sometimes the entire workflow just hangs. Also, the workflow editor seems to be having issues, like it allows connections between outputs and inputs of entirely different datatypes. Has anyone seen any issues like this? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Plugins
Over next couple days, I'll take a look at what's in lib/galaxy/web/base/pluginframework.py and see if there is a way to refactor it with my stuff in order to get a general plugin framework (galaxy.pluginframework) as proposed by James. Sorry for the delay. Teaching a math class this semester is really eating up my time. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Department Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Tuesday, September 3, 2013 at 12:45 PM, Carl Eberhard wrote: It is web-specific and only handles the mako/static serving cases. I think the only areas we were generalizing at this point were the directory structures and main configuration. Agreed on more generalization and the hook system. Michael's stuff is a good direction to move in. I know John was looking to use it for dynamic job destinations and dynamic toolbox filters. I'd like to see some lab/community discussion on (even broad stroke) functional requirements and use cases. On Tue, Sep 3, 2013 at 12:27 PM, James Taylor ja...@jamestaylor.org (mailto:ja...@jamestaylor.org) wrote: Carl, is what you have now totally web specific? It would be great to have a general plugin framework (galaxy.pluginframework) which the web stuff could perhaps extend? On Tue, Sep 3, 2013 at 10:56 AM, Carl Eberhard carlfeberh...@gmail.com (mailto:carlfeberh...@gmail.com) wrote: Heya, Michael Right now our only plugin code is located in lib/galaxy/web/base/pluginframework.py. It's only purpose so far is to: - allow locally modified/created code to serve mako templates and static pages/resources from paste. - serve as a super class for the visualization framework. I think your git repo has some great ideas. Carl On Fri, Aug 30, 2013 at 2:25 PM, Michael Cotterell mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) wrote: Is this plugin code hosted anywhere (like in central)? On Monday, August 26, 2013, James Taylor wrote: Michael, Carl has been working on a plugin framework for Galaxy, and John Chilton has made some improvements. It would be great if you could look over what each other has done and come up with some suggestions. I personally am in favor of a very generic hook system where a hook can be defined anywhere in the code and then the user can provide any callable. I'm fond of the way sup implemented hooks, see the use of HookManager here: https://github.com/sup-heliotrope/sup/blob/develop/lib/sup/time.rb Your approach is quite similar and I like it. This makes it easy to have a script that can list all available hooks with their documentation. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Fri, Aug 23, 2013 at 10:29 AM, Michael E. Cotterell mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) wrote: James, Has there been any more talk about this? I'd be willing to work in my reference implementation into central as a pull request, but I don't want to do that unless there is an agreed upon roadmap or something. Thanks! Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Department Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Tuesday, July 16, 2013 at 2:08 PM, Michael E. Cotterell wrote: James, That's exactly what I was thinking. In my opinion, hooks are the way to go (as seen in my example), but if there's a better way then I'm game for that to. I just want to be able to extend Galaxy without modifying Galaxy itself. Thanks! Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu)
Re: [galaxy-dev] Geek question re HTTP POST and Galaxy
hi Jeremy, in the histories controller, I could find no example to nicely obtain the PUT and POST content payload. Am I missing it? So I think a bad answer to my question is the following code. But I really don't like my _get_payload() hack reaching into the trans.environ object. Is there a better way to do this? The short answer is that Galaxy has middleware that unpacks API request content and puts them into parameters. For example, the URL /api/histories/2f3f601bb97a6a89 calls histories/index with id=2f3f601bb97a6a89 The data from a POST or PUT appears as payload for create/update methods. in buildapp.py // I also don't like the fact that it needs five separate calls. this is the sort of design pattern that should be very easy. webapp.mapper.connect(/api/assets/, controller=medbook, action=assets_create, conditions=dict(method=[POST])) webapp.mapper.connect(/api/assets/, controller=medbook, action=assets_read, conditions=dict(method=[GET])) webapp.mapper.connect(/api/assets/*url, controller=medbook, action=assets_read, conditions=dict(method=[GET])) webapp.mapper.connect(/api/assets/*url, controller=medbook, action=assets_update, conditions=dict(method=[PUT])) webapp.mapper.connect(/api/assets/*url, controller=medbook, action=assets_delete, conditions=dict(method=[DELETE])) Something like this should work for all five: webapp.mapper.resource( 'asset', 'assets', path_prefix='/api' ) In general, your best bet is to replicate an existing API controller as closely as you can. J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] [GSoC2013] Update
I wrote a very short update here: http://galaxy-gsoc2013.blogspot.in/2013/09/penultimate-update.html Saket ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Functional tests results in Invalid HTTP return code 404/500, allowed codes: 200
Dear Galaxy hackers, I'm having trouble with the tool test framework. I have a local installation of a Galaxy Tool Shed (latest stable from the repository) on my Linux server. I have an Apache proxy to my tool shed (per http://wiki.galaxyproject.org/HostingALocalToolShed). The Tool Shed seems to work fine. I'm able to create and maintain repositories and I'm able to install tools from the Tool Shed from remote Galaxy instances. Now I want to run the functional tests of the tools in the Tool Shed. I do that by following the instructions on the wiki (http://wiki.galaxyproject.org/AutomatedToolTests). It comes down to the following commands: -- cd /opt/galaxy/galaxy-dist/ export GALAXY_INSTALL_TEST_TOOL_SHED_API_KEY=2aeecfe2d9f28d0c5b6959a41886b5fd export GALAXY_INSTALL_TEST_TOOL_SHED_URL=http://toolshed.galaxy.domain.org/ python lib/tool_shed/scripts/check_repositories_for_functional_tests.py tool_shed_wsgi.ini sh install_and_test_tool_shed_repositories.sh -- The second script (install_and_test_tool_shed_repositories.sh) fails with the following error: == FAIL: install_repository_usearch (install_and_test_tool_shed_repositories.functional.test_install_repositories.TestInstallRepository_usearch) Install the repository usearch from http://toolshed.galaxy.domain.org/. -- Traceback (most recent call last): File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py, line 48, in test_install_repository self.do_installation( repository_dict ) File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py, line 17, in do_installation self.install_repository( repository_info_dict ) File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py, line 61, in install_repository self.visit_url( '%s/repository/preview_tools_in_changeset?%s' % ( tool_shed_url, preview_params ) ) File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py, line 112, in visit_url ( return_code, url, ', '.join( str( code ) for code in allowed_codes ) ) AssertionError: Invalid HTTP return code 404 for http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12changeset_revision=095baaf5d877, allowed codes: 200 -- (I've changed twilltestcase.py to also print the URL that is being browsed.) This is were it get's weird. The HTTP 404 response code indicates that the page was not found. However, when I browse to that URL, it works fine! What's more, when I try to access that page with `wget` from the same machine the HTTP response code is 200: -- wget http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12changeset_revision=095baaf5d877 ... HTTP request sent, awaiting response... 200 OK ... -- When I use the Tool Shed URL http://localhost:9009/ instead, I get a similar error, only the return code is 500 instead of 404. I suspect that the HTTP return code returned by the Python module that returns it (Mechanize?) is invalid. I found a note about this error on the wiki (http://wiki.galaxyproject.org/Admin/Running%20Tests): Note: If you have another version of paste installed in your PYTHONPATH, it can cause problems running the functional tests. Many tests will error (instead of passing/failing) because of an HTML code 500 where a 200 was expected. There are two workarounds for this: uninstall paste or use virtualenv. The thing is, I do not have another version of Paste installed. What can I do to fix this error? Regards, Serrano Pereira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] workflow issues in local Galaxy....
Hi all, So we use Galaxy to teach our bioinformatics courses and we have an install of Galaxy in the Amazon cloud that we have customized with various tools. However, we don't touch the Galaxy code base itself. Recently, we've been getting errors in creating and running workflows. We are getting locked database errors, sometimes the steps seem to run out of order, and sometimes the entire workflow just hangs. Also, the workflow editor seems to be having issues, like it allows connections between outputs and inputs of entirely different datatypes. Has anyone seen any issues like this? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Can the API add elements to a repeat parameter?
Hi, I'm interested on running workflows with tool definitions containing repeat inputs and arguments. I was wondering if there is a way to not have to set a fix number of elements ahead of time in the workflow. Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Functional tests results in Invalid HTTP return code 404/500, allowed codes: 200
Serrano, The GALAXY_INSTALL_TEST_TOOL_SHEDS_CONF environment variable should be set to an XML file that specifies the tool shed you're installing from. I've updated the wiki page to be clearer on that point. --Dave B. On 09/16/2013 12:32 PM, Serrano Pereira wrote: Dear Galaxy hackers, I'm having trouble with the tool test framework. I have a local installation of a Galaxy Tool Shed (latest stable from the repository) on my Linux server. I have an Apache proxy to my tool shed (per http://wiki.galaxyproject.org/HostingALocalToolShed). The Tool Shed seems to work fine. I'm able to create and maintain repositories and I'm able to install tools from the Tool Shed from remote Galaxy instances. Now I want to run the functional tests of the tools in the Tool Shed. I do that by following the instructions on the wiki (http://wiki.galaxyproject.org/AutomatedToolTests). It comes down to the following commands: -- cd /opt/galaxy/galaxy-dist/ export GALAXY_INSTALL_TEST_TOOL_SHED_API_KEY=2aeecfe2d9f28d0c5b6959a41886b5fd export GALAXY_INSTALL_TEST_TOOL_SHED_URL=http://toolshed.galaxy.domain.org/ python lib/tool_shed/scripts/check_repositories_for_functional_tests.py tool_shed_wsgi.ini sh install_and_test_tool_shed_repositories.sh -- The second script (install_and_test_tool_shed_repositories.sh) fails with the following error: == FAIL: install_repository_usearch (install_and_test_tool_shed_repositories.functional.test_install_repositories.TestInstallRepository_usearch) Install the repository usearch from http://toolshed.galaxy.domain.org/. -- Traceback (most recent call last): File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py, line 48, in test_install_repository self.do_installation( repository_dict ) File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py, line 17, in do_installation self.install_repository( repository_info_dict ) File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py, line 61, in install_repository self.visit_url( '%s/repository/preview_tools_in_changeset?%s' % ( tool_shed_url, preview_params ) ) File /opt/galaxy/galaxy-dist/test/install_and_test_tool_shed_repositories/base/twilltestcase.py, line 112, in visit_url ( return_code, url, ', '.join( str( code ) for code in allowed_codes ) ) AssertionError: Invalid HTTP return code 404 for http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12changeset_revision=095baaf5d877, allowed codes: 200 -- (I've changed twilltestcase.py to also print the URL that is being browsed.) This is were it get's weird. The HTTP 404 response code indicates that the page was not found. However, when I browse to that URL, it works fine! What's more, when I try to access that page with `wget` from the same machine the HTTP response code is 200: -- wget http://toolshed.galaxy.domain.org//repository/preview_tools_in_changeset?repository_id=3651c288c4818b12changeset_revision=095baaf5d877 ... HTTP request sent, awaiting response... 200 OK ... -- When I use the Tool Shed URL http://localhost:9009/ instead, I get a similar error, only the return code is 500 instead of 404. I suspect that the HTTP return code returned by the Python module that returns it (Mechanize?) is invalid. I found a note about this error on the wiki (http://wiki.galaxyproject.org/Admin/Running%20Tests): Note: If you have another version of paste installed in your PYTHONPATH, it can cause problems running the functional tests. Many tests will error (instead of passing/failing) because of an HTML code 500 where a 200 was expected. There are two workarounds for this: uninstall paste or use virtualenv. The thing is, I do not have another version of Paste installed. What can I do to fix this error? Regards, Serrano Pereira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the
