[galaxy-dev] Installing SOAPdenovo tools
Dear Devs I was wondering if anyone on this list has experience of installing the SOAPdenovo tools from GigaGalaxy? The tool sources are hosted on github at https://github.com/gigascience/galaxy-bgisoap and the README says that the tools are hosted on the GigaToolshed at http://gigatoolshed.net/, but this doesn't seem to be accessible. I don't think manual installation from the github sources is practical. Does anyone have any suggestions or experience of installing these tools into their local Galaxy instances? Thanks! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: %(web_page_url)s To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Re: Workflow assigning columns on output causes "Uncaught exception in exposed API method"
Hello Marius
Thanks for looking into this - I've tried the workaround you suggested and the
workflow now seems to start okay.
Thanks also for the fix in the PR, much appreciated!
Best wishes
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
From: Marius van den Beek [m.vandenb...@gmail.com]
Sent: Thursday, May 23, 2019 2:18 PM
To: Peter Briggs
Cc: galaxy-dev@lists.galaxyproject.org; Ian Donaldson
Subject: Re: [galaxy-dev] Workflow assigning columns on output causes "Uncaught
exception in exposed API method"
Hi Peter,
thanks for the report, I've opened a pull request to fix this at
https://github.com/galaxyproject/galaxy/pull/8023.
We'll backport this to 18.09 after acceptance.
As a (terrible) workaround you could download the workflow and change the
section that looks like
```
"action_type": "ColumnSetAction",
"output_name": "out_file1",
"action_arguments": {
"chromCol": 1,
"endCol": 3,
"startCol": 2
}
```
to
```
"action_type": "ColumnSetAction",
"output_name": "out_file1",
"action_arguments": {
"chromCol": "1",
"endCol": "3",
"startCol": "2"
}
```
(the "output_name" value may be different for each tool) and then upload it
back to the instance.
Hope that helps,
Marius
On Wed, 22 May 2019 at 10:36, Peter Briggs
mailto:peter.bri...@manchester.ac.uk>> wrote:
Dear Devs
We've encountered a problem trying to run a workflow on our local Galaxy
instance, which is resulting in an "uncaught exception in exposed API method"
message displayed in the Galaxy window.
The most trivial workflow exhibiting the problem consists of a single 'cut'
task, where the output of the task is configured to change the datatype to
"interval", and to explicitly assign the chrom, start and end columns to 1,2
and 3. Attempts to run this workflow generate the API exception.
If the workflow is updated so that the column assignments are no longer
attempted then the workflow is able to start without apparent problems.
Our local Galaxy instance is a bit out of data (18.09) but I have been able to
reproduce this error on Galaxy main so it looks like it also affects later
versions. The failing workflow can be accessed on main via:
https://usegalaxy.org/u/pjbriggs/w/cut-workflow-reassign-columns-fails-with-uncaught-api-exception
The same workflow with the column assignment removed is accessible via:
https://usegalaxy.org/u/pjbriggs/w/cut-workflow-no-column-reassignment
(Main offers two variants of the 'cut' tool - as far as I can tell this problem
happens with both.)
Apologies if this has already been reported elsewhere - any suggestions for a
workaround appreciated.
Thanks,
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
%(web_page_url)s
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
%(web_page_url)s
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
[galaxy-dev] Workflow assigning columns on output causes "Uncaught exception in exposed API method"
Dear Devs We've encountered a problem trying to run a workflow on our local Galaxy instance, which is resulting in an "uncaught exception in exposed API method" message displayed in the Galaxy window. The most trivial workflow exhibiting the problem consists of a single 'cut' task, where the output of the task is configured to change the datatype to "interval", and to explicitly assign the chrom, start and end columns to 1,2 and 3. Attempts to run this workflow generate the API exception. If the workflow is updated so that the column assignments are no longer attempted then the workflow is able to start without apparent problems. Our local Galaxy instance is a bit out of data (18.09) but I have been able to reproduce this error on Galaxy main so it looks like it also affects later versions. The failing workflow can be accessed on main via: https://usegalaxy.org/u/pjbriggs/w/cut-workflow-reassign-columns-fails-with-uncaught-api-exception The same workflow with the column assignment removed is accessible via: https://usegalaxy.org/u/pjbriggs/w/cut-workflow-no-column-reassignment (Main offers two variants of the 'cut' tool - as far as I can tell this problem happens with both.) Apologies if this has already been reported elsewhere - any suggestions for a workaround appreciated. Thanks, Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: %(web_page_url)s To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Re: URL prefix doesn't work properly in Galaxy v18.09
Replying to my own message in case it's of use to someone else:
It looks like this can be fixed by commenting out the 'target:
"__use_router__"' lines in client/galaxy/scripts/layout/menu.js i.e.
diff --git a/client/galaxy/scripts/layout/menu.js
b/client/galaxy/scripts/layout/menu.js
index 2dc1b41098..2d4c1db8c7 100644
--- a/client/galaxy/scripts/layout/menu.js
+++ b/client/galaxy/scripts/layout/menu.js
@@ -46,7 +46,8 @@ var Collection = Backbone.Collection.extend({
tooltip: _l("Chain tools into workflows"),
disabled: !Galaxy.user.id,
url: "workflows/list",
-target: "__use_router__"
+//PJB remove target as it seems to break when using proxy-prefix
+//target: "__use_router__"
});
//
@@ -61,7 +62,8 @@ var Collection = Backbone.Collection.extend({
{
title: _l("Create Visualization"),
url: "visualizations",
-target: "__use_router__"
+//PJB remove target as it seems to break when using
proxy-prefix
+//target: "__use_router__"
},
{
title: _l("Interactive Environments"),
(I needed to recompile the packed Javascript afterwards by running the
common_startup.sh script before the changes appeared in the web interface.)
This appears to fix the links to "Workflows" and "Visualisations" in the
masthead of the main page.
HTH
Best wishes
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
From: Peter Briggs [peter.bri...@manchester.ac.uk]
Sent: Thursday, May 16, 2019 2:01 PM
To: Ping Luo; galaxy-dev
Subject: [galaxy-dev] Re: URL prefix doesn't work properly in Galaxy v18.09
Hello Ping (and Galaxy-devs)
Did this get resolved? We are still on 18.09 and serving our production
instance with a proxy prefix, and one of my users has hit this issue when
trying to access the "Workflows" tab from the main page.
As Ping says, it appears that Galaxy is inserting an extra copy of the prefix
into the target URL - if the user removes this manually then the page loads
okay.
Does anyone know, is there a fix for this in 18.09?
Thanks!
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Ping
Luo [luop0...@gmail.com]
Sent: Monday, December 17, 2018 9:04 PM
To: galaxy-dev
Subject: [galaxy-dev] URL prefix doesn't work properly in Galaxy v18.09
I configured URL prefix in galaxy 18.09. It has problems of resolving some of
the links, including:
workflow
visualize -> create visualization
shared data -> workflows
shared data -> visualizations
I will use workflow as an example to illustrate what happens. When the mouse
cursor is put on workflow, the URL shown in at the left bottom of the browser
shows
https://xxx.xxx.tamu.edu/galaxy/workflows/list
But after I clicked 'workflow', the real URL shown in the address bar is
actually
https://xxx..tamu.edu/galaxy/galaxy//workflows/list?__identifer=sk0xsywiqrs
Of course, there is no such a page and the browser shows a blank page. If I
manually remove one galaxy in the URL, the page is correctly shown.
Beside the four mentioned above, other links work fine. If galaxy is configured
without prefix, then there is no problem.
I also tested v18.05. With all the configuration being the same, this version
doesn't have the problem with prefix like v18.09 does. So I think this is a bug
in v18.09 with URL prefix.
Ping
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
%(web_page_url)s
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
%(web_page_url)s
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
[galaxy-dev] Re: URL prefix doesn't work properly in Galaxy v18.09
Hello Ping (and Galaxy-devs) Did this get resolved? We are still on 18.09 and serving our production instance with a proxy prefix, and one of my users has hit this issue when trying to access the "Workflows" tab from the main page. As Ping says, it appears that Galaxy is inserting an extra copy of the prefix into the target URL - if the user removes this manually then the page loads okay. Does anyone know, is there a fix for this in 18.09? Thanks! Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Ping Luo [luop0...@gmail.com] Sent: Monday, December 17, 2018 9:04 PM To: galaxy-dev Subject: [galaxy-dev] URL prefix doesn't work properly in Galaxy v18.09 I configured URL prefix in galaxy 18.09. It has problems of resolving some of the links, including: workflow visualize -> create visualization shared data -> workflows shared data -> visualizations I will use workflow as an example to illustrate what happens. When the mouse cursor is put on workflow, the URL shown in at the left bottom of the browser shows https://xxx.xxx.tamu.edu/galaxy/workflows/list But after I clicked 'workflow', the real URL shown in the address bar is actually https://xxx..tamu.edu/galaxy/galaxy//workflows/list?__identifer=sk0xsywiqrs Of course, there is no such a page and the browser shows a blank page. If I manually remove one galaxy in the URL, the page is correctly shown. Beside the four mentioned above, other links work fine. If galaxy is configured without prefix, then there is no problem. I also tested v18.05. With all the configuration being the same, this version doesn't have the problem with prefix like v18.09 does. So I think this is a bug in v18.09 with URL prefix. Ping ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: %(web_page_url)s To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] how to configure sub-uri in galaxy.yml
Hello Ping If you're using the YAML config then there isn't an equivalent of the "[filter:proxy-prefix]" section from the INI version of the config - this section doesn't seem to be required any more. So the two lines you already have in the 'galaxy' section of the YAML file should be sufficient i.e. galaxy: #... filter-with: proxy-prefix cookie_path: '/galaxy' However you also need to make sure that you have the correct options in the 'uwsgi' section of the YAML config file, and that your Apache configuration is set correctly to handle the proxy prefix. The relevant documentation for these is here (for 18.09): https://docs.galaxyproject.org/en/release_18.09/admin/apache.html#serving-galaxy-at-a-url-prefix HTH Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Ping Luo [luop0...@gmail.com] Sent: Wednesday, December 12, 2018 6:35 AM To: galaxy-dev Subject: [galaxy-dev] how to configure sub-uri in galaxy.yml I am trying to configure Galaxy v18.09 wtih sub-uri. In prior 18.01, I use this configuration in conjunction with Apache rewrite rules and it works well: [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /msgalaxy [app:main] filter-with = proxy-prefix cookie_path = '/galaxy' In galaxy.yml, I know I need to uncomment and add the following filter-with: proxy-prefix cookie_path: '/galaxy' However, I don't know where and how to add filter:proxy-prefix. The comment in config.yml says: # If running behind a proxy server and Galaxy is served from a # subdirectory, enable the proxy-prefix filter and set the prefix in # the [filter:proxy-prefix] section above. which is for galaxy.ini file. An example on how to do it would be much appreciated. Ping ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] From Paste to uWSGI
Hello Jochen I'm not aware of a tutorial but here are some observations from my (very) recent experiences upgrading a local paste-based 17.09 instance to a uwsgi-based 18.05 configuration. It's not comprehensive because the configuration will depend on decisions you need to make based on your local setup, but it might be helpful as a starting point. Before beginning: -- Commit to migrating from galaxy.ini to galaxy.yml (if not done previously) Even though its possible to stick with the old galaxy.ini file, it seems best to move to the new format - my feeling is, probably the migration will have to happen at some point anyway, and also the documentation is more focussed on the newer galaxy.yml version. -- Decide on how you're going to handle job submission (essentially, how you're going to replace the old job handler processes). There are a number of possibilities outlined in the documentation here: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#deployment-options It seems important to figure this out at the beginning because it impacts a lot of the configuration specifics that follow. For our setup case "uwsgi + mules" looked like the most appropriate: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi-for-web-serving-with-mules-as-job-handlers Then once you're ready to start (nb backing up the database etc before starting are implicitly assumed!): -- Update the Galaxy source code The standard recipe for this seems to work okay i.e. git fetch origin && git checkout release_18.05 && git pull --ff-only origin release_18.05 (You'll probably also need to run scripts/common_startup.sh to update the Python packages, and scripts/manage_db.py to update the database schema.) -- Create galaxy.yml and configure the "galaxy" section: You can make a galaxy.yml from the sample version and go through the "galaxy" section replicating the settings from galaxy.ini. (The settings in the "uwsgi" section can wait at this stage.) Don't forget to move galaxy.ini out of the way once you're done. -- Configure the "uwsgi" section of galaxy.yml: The options for various scenarios are given in the "Deployment/uwsgi" section: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi but the required settings seem to be spread out between different sections so (for me) it felt like it took some piecing together to get it to work when I first tried this. (Also I was thrown a little by the "socket" parameter not having a placeholder in sample galaxy.yml.) The core uwsgi options are here: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#configuration-common-to-all-uwsgi-deployment-styles then you need to set other options dependent on: 1. how job submission is handled (e.g. options for the "uwsgi + mules" setup in my case) 2. how you're proxying Galaxy (for Apache or nginx it looks like it's just setting the uwsgi "socket" option, unless you're serving Galaxy from a URL prefix in which case there are a few more options needed - these are covered in the documentation.) -- Fix your job_conf.xml file If you have an existing job_conf.xml file in your config directory then you might need to update that too (specifically the "handlers" section) depending on the job submission handling you've decided to use. -- Use supervisor to manage your uwsgi processes The docs for this are here: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#starting-and-stopping Essentially supervisor replaces "run.sh", so this becomes how you start and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces "GALAXY_RUN_ALL=1 sh run.sh --daemon".) I think this is quite straightforward from the docs but again you need to tailor it for how you're handling job submission (e.g the "uwsgi + mules" doesn't need any handlers defining, but it looks like other strategies needs extra stuff defining here). -- Set up the proxying Assuming you're proxing via either Apache or nginx (we're using nginx), these are covered in the relevant sections: https://docs.galaxyproject.org/en/release_18.05/admin/apache.html https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html The main difference I noticed for nginx is that you no longer need to use the "upstream"/"proxy_pass" directives, as nginx can talk directly to uwsgi using a "uwsgi_pass" directive that connects to the socket defined in the "uwsgi" section of galaxy.yml previously. (There's also a "gotcha" if you're nginx configuration also tries to serve "static/scripts/packed", which needs to be removed - this is noted in the "breaking changes" for the 18.05 docs but is easy to miss.) Hopefully this is some help. There's lots of document
[galaxy-dev] Unable to install tools from toolshed/mercurial issue in release_18.05 instance
Dear devs I've encountered what appears to be a subtle bug with release 18.05, which breaks the installation of tools from the toolshed, and appears to be a result of not having mercurial (hg) available in /usr/bin on the system that Galaxy is installed on (in this case Scientific Linux 6.5). When attempting to install a tool (e.g. devteam/fastqc) from the main toolshed via the admin interface, after clicking "install" the tool installation status goes immediately to "Error". The tool repository isn't cloned to "shed_tools" and no dependencies are installed. I've been unable to find any error messages in the logs. However, attempting to install via the API does return the message: Error cloning repository: [Errno 2] No such file or directory which comes from the "clone_repository" function in lib/tool_shed/util/hg_util.py (when something goes wrong with the "hg clone ..." command). Installing Mercurial 1.3 via yum on the server and attempting tool installation again gives a slightly different error via the API: Error cloning repository: Command '['hg', 'clone', '-r', u'17', u'https://toolshed.g2.bx.psu.edu/repos/devteam/fastqc', u'/XX/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc']' returned non-zero exit status 255 Output was: abort: No such file or directory: /XX/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc Uninstalling the system Mercurial and instead installing version 3.7.3 and making a link from /usr/bin/hg seems to fix the problem, and tools can be installed without problems. I couldn't find any evidence of this being reported before, and I don't know if I've missed some configuration detail which means that Galaxy isn't picking up hg from its virtualenv instead of /usr/bin. Has anyone else encountered this problem? and is there a fix/workaround for it (other than horrible links to /usr/bin/hg)? Any advice gratefully received! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Permanently deleting users from a Galaxy instance
Hello Eric Thanks for the fulsome reply - that looks awesome! Looks like an upgrade to 18.05 is needed for our servers. Thanks for your efforts on this, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: h...@hx42.org [h...@hx42.org] Sent: Monday, June 11, 2018 10:20 AM To: Peter Briggs Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Permanently deleting users from a Galaxy instance Hi Peter, On 2018-06-11, Peter Briggs wrote: > Hello > > I'm wondering if there is an option to permanently delete a user account from > a Galaxy instance, for example to comply with GDPR (for those of us running > public Galaxy instances). I implemented the `beta_gdpr_mode` in Galaxy 18.05. Please find the pull request here: https://github.com/galaxyproject/galaxy/pull/6069 In case you are interested how this functionality was implemented And it is documented in the release notes here: https://docs.galaxyproject.org/en/release_18.05/releases/18.05_announce.html#highlights And in the admin documentation here: https://docs.galaxyproject.org/en/master/admin/special_topics/gdpr_compliance.html (but that documentation needs to be updated.) > The admin interface provides "delete" and "purge" options for user accounts, > but neither of these seems to permanently remove an account - the deleted > accounts are still visible and can be undeleted (at least, in Galaxy 17.09). > It's also unclear to me what "purge" does in this case - I wasn't able to > find any documentation on these user management options. I believe it's the same distinction made with histories - deleted = "cleanup in some time" - purged = "fine to cleanup now" > As I understand it, under GDPR a person can request to have all their > personal data removed from a database, but neither of the above options would > be sufficient to leave the database compliant with GDPR in this case (as at > the very least the email address - which I understand constitutes personal > information in this context - would remain in the database). Yes, our current opinion is that only the email address + username + any addresses on file must be wiped. This feature is available to you when you turn on this mode. > Is there any way within Galaxy to truly permanently remove a user account? > > Thanks in advance for any advice on this, > > Best wishes > > Peter > > -- > Peter Briggs peter.bri...@manchester.ac.uk > Bioinformatics Core Facility University of Manchester > B.1083 Michael Smith Bldg Tel: (0161) 2751482 > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Permanently deleting users from a Galaxy instance
Hello I'm wondering if there is an option to permanently delete a user account from a Galaxy instance, for example to comply with GDPR (for those of us running public Galaxy instances). The admin interface provides "delete" and "purge" options for user accounts, but neither of these seems to permanently remove an account - the deleted accounts are still visible and can be undeleted (at least, in Galaxy 17.09). It's also unclear to me what "purge" does in this case - I wasn't able to find any documentation on these user management options. As I understand it, under GDPR a person can request to have all their personal data removed from a database, but neither of the above options would be sufficient to leave the database compliant with GDPR in this case (as at the very least the email address - which I understand constitutes personal information in this context - would remain in the database). Is there any way within Galaxy to truly permanently remove a user account? Thanks in advance for any advice on this, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Problems running reporting app with reports.yml using --daemon
Dear devs I'm trying to update my ansible playbooks for deploying Galaxy and I'm having problems when trying to run the reporting app in release_17.09 using a 'reports.yml' file (rather than 'reports.ini'), when specifying the --daemon option. Running without any arguments, the application appears to start okay - but if I do sh ./run_reports.sh --daemon then it exits with the error: /X/galaxy/.venv/bin/python: unrecognized option '--safe-pidfile' getopt_long() error This seems to be coming from the invocation of uwsgi within the run_reports.sh script, and looks like an issue with Galaxy's version of uWSGI: version 2.0.15 (as specified in the requirements.txt file) doesn't support the '--safe-pidfile' option (that needs uWSGI 2.0.16). Editing requirements.txt to use 2.0.16 seems to address this issue (as in, --daemon no longer fails on startup). Has anyone else encountered this issue? My ultimate goal is to be able to serve the uwsgi-based reports app via an nginx proxy (which used to work with the old i.e. non-uwsgi reports.ini-configured version). Thanks for your help, Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Ensuring Python 2.7 when resolving conda dependencies for tool
Hello Marius and Peter Thanks for all the info & suggestions, that was really helpful and I can see what's happening now (and why it's not working in my specific case). If you're interested in the details: -- If the tool requires python==2.7 and biopython=1.65 only, then as Marius says the dependencies are resolved and Galaxy creates a "mulled" environment with the correct package versions installed. -- In my case however I have some additional requirements defined, which are not yet in bioconda or the other channels (at this stage they've just been built locally, while I try to get everything updated). In this case conda fails to resolve some of the dependencies when trying to create the mulled environment, and bails out - instead it creates individual environments (e.g. __python@2.7, __biopython@1.65 etc) just for those packages it can find. But this fallback "standalone" biopython environment doesn't have the python==2.7 requirement transferred in, hence it defaults to python3. I guess I didn't have this issue previously when migrating other tools (even when there were local dependencies) as there was no python dependency in those cases. So it looks like I need to sort out the missing dependencies first, then come back to testing the tool. Thanks again both for all your help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Peter Cock [p.j.a.c...@googlemail.com] Sent: Monday, March 05, 2018 5:04 PM To: Marius van den Beek Cc: Peter Briggs; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Ensuring Python 2.7 when resolving conda dependencies for tool Oh yeah - thanks, I can see them in the default channel which does have far older versions of Biopython packaged - currently 1.65 is on page 5: https://anaconda.org/anaconda/biopython/files?sort=basename_order=desc=5 Peter On Mon, Mar 5, 2018 at 5:00 PM, Marius van den Beek <m.vandenb...@gmail.com> wrote: > It's coming from the defaults channel, I didn't check if this is actually > working, > it may very well not be compatible with the remaining packages form > conda-forge. > AFAIK you can't specify packages to install via the regular conda commands > (though you can use pip in a conda environemnt to do this manually). > > Hope that helps, > Marius > > On 5 March 2018 at 17:51, Peter Cock <p.j.a.c...@googlemail.com> wrote: >> >> I stand corrected. Looking closer, there are conda packages >> for both Python 2 and 3, e.g. >> >> https://anaconda.org/conda-forge/biopython/files >> >> However, in this case you're asking for an older Biopython >> which has to date not been packaged in conda-forge or >> bioconda, so I presume in Marius example it comes from >> PyPI via pip? >> >> >> Peter >> >> On Mon, Mar 5, 2018 at 4:46 PM, Marius van den Beek >> <m.vandenb...@gmail.com> wrote: >> > This should actually work properly if you install the dependencies via >> > the Manage dependencies page in the admin menu or if you install the >> > tool >> > via the tool shed. >> > This translates more or less to the following conda command: >> > ``` >> > $ conda create -n mulled-v1 python=2.7 biopython=1.65 -c iuc -c >> > bioconda -c conda-forge -c defaults >> > Fetching package metadata ... >> > Solving package specifications: . >> > >> > Package plan for installation in environment >> > /Users/mvandenb/miniconda3/envs/test_resolution: >> > >> > The following NEW packages will be INSTALLED: >> > >> > biopython: 1.65-np19py27_0 >> > ca-certificates: 2018.1.18-0 conda-forge >> > certifi: 2018.1.18-py27_0 conda-forge >> > intel-openmp:2018.0.0-h8158457_8 >> > libgfortran: 3.0.1-h93005f0_2 >> > mkl: 2018.0.1-hfbd8650_4 >> > ncurses: 5.9-10 conda-forge >> > numpy: 1.9.3-py27hb3dd696_3 >> > openssl: 1.0.2n-0 conda-forge >> > pip: 9.0.1-py27_1 conda-forge >> > python: 2.7.14-4 conda-forge >> > readline:7.0-0conda-forge >> > setuptools: 38.5.1-py27_0conda-forge >> > sqlite: 3.20.1-2 conda-forge >> > tk: 8.6.7-0 conda-forge >> > wheel: 0.30.0-py27_2conda-forge >> > zlib:1.2.11-0
[galaxy-dev] Ensuring Python 2.7 when resolving conda dependencies for tool
Hello I'm in the process of updating our local Galaxy tools to use conda dependency resolution, and I've hit a snag with a tool that requires Python 2.7 along with the Python 2.7-compatible version of Biopython 1.65. I'd assumed that if I explicitly used the following in the "requirements" section of the tool XML: python biopython that the biopython install would respect the specified Python version, and that the command execution environment would be based on Python 2.7. But in practice it appears I'm getting Python3 as I'm seeing errors: File "/galaxy-tools/tools/pal_finder/pal_filter.py", line 129 print "\n~~" ^ SyntaxError: Missing parentheses in call to 'print' This is happening both when running the tool tests via the planemo tests, and also when I install the tool from a local toolshed instance. Can anyone advise how to coerce the Python dependencies to be 2.7-compatible? Thanks in advance for any help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Running toolshed on localhost is broken in release_17.09?
Hello Nate Thanks for looking into this so quickly - I've manually applied the patch in your PR for toolshed.yml.sample and that works perfectly, thanks! Re picking up older .ini config files for a toolshed instance: what I was doing was spinning up a completely new pair of Galaxy and toolshed instances on my local machine, just for testing purposes. So in this case there was no pre-existing .ini files to be picked up. Hope that makes sense - thanks again for your help, and the patch. Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Nate Coraor [n...@bx.psu.edu] Sent: Wednesday, February 14, 2018 4:19 PM To: Peter Briggs Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Running toolshed on localhost is broken in release_17.09? Hi Peter, This might be due to the switch from Paste to uWSGI for reports and tool shed applications in 17.09. I do see that when using the 17.09 default configuration (uWSGI with config/tool_shed.yml.sample), the static content is not configured to be served, so I submitted a fix for that here: https://github.com/galaxyproject/galaxy/pull/5534 However, if you have an existing tool_shed.ini, run_tool_shed.sh is supposed to continue to use Paste, and since run_tool_shed.sh (presumably) decided to start with uWSGI instead of Paste, it did not find your tool_shed.ini. Is it located somewhere other than config/tool_shed.ini? --nate On Wed, Feb 14, 2018 at 9:34 AM, Peter Briggs <peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>> wrote: Hello I'm trying to run a toolshed instance on localhost to test some local tool changes - this works okay in release_17.05, but setting one up using release_17.09 appears to be broken out of the box, at least for me. In release_17.09, after configuration connecting to http://127.0.0.1:9009/ brings up a version which isn't usable - the styles are missing and it it's not possible to navigate, register/login etc. Uncommenting lines like # Serving static files (needed if running standalone) static_enabled: True in the config/tool_shed.yml file don't seem to make any difference. I didn't find anything in the Galaxy Github issue list about this, so has anyone else observed this behaviour? For now I've just gone back to release_17.05 for my local toolshed needs. Thanks Peter -- Peter Briggs peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk> Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Running toolshed on localhost is broken in release_17.09?
Hello I'm trying to run a toolshed instance on localhost to test some local tool changes - this works okay in release_17.05, but setting one up using release_17.09 appears to be broken out of the box, at least for me. In release_17.09, after configuration connecting to http://127.0.0.1:9009/ brings up a version which isn't usable - the styles are missing and it it's not possible to navigate, register/login etc. Uncommenting lines like # Serving static files (needed if running standalone) static_enabled: True in the config/tool_shed.yml file don't seem to make any difference. I didn't find anything in the Galaxy Github issue list about this, so has anyone else observed this behaviour? For now I've just gone back to release_17.05 for my local toolshed needs. Thanks Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] "Edit attributes" for interval dataset not consistent with documentation
Dear Devs One of my local users has noticed that the "edit attributes" function for "interval"-type datasets is not consistent with the documentation at: https://galaxyproject.org/learn/datatypes/#interval This states that only CHROM, START, and END are required, with the implication that other fields are optional. However when editing the attributes for a dataset, if either of "Strand" or "Name" are not set to an explicit column number then "Save attributes" fails with the message: "Error occurred while saving. Please fill all the required fields and try again." This is true on both our local Galaxy running release_17.09 and also on Galaxy main. Is this a bug in the documentation, or the implementation? Thanks, Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Problems installing conda dependencies for seq2hla tool
Just to follow up on my own question, in case it's helpful to anyone else: Re the specific problem with not being able to install the seq2HLA 2.2 dependency via conda: in the end I build my own version and made it available to Galaxy as "Galaxy package" (https://docs.galaxyproject.org/en/master/admin/dependency_resolvers.html#galaxy-packages-dependency-resolver), which seemed to work okay. Re updating to conda 4.3.24: after the update this gave me problems with at least one dependency, due to the "--offline bug" described here: https://github.com/galaxyproject/galaxy/issues/5187 I applied Norbert Eichner's fix in PR #5233 (https://github.com/galaxyproject/galaxy/pull/5233) and reinstalled the conda dependency via "Manage Tool Dependencies" in the "Admin" interface, which seemed to fix that problem. It's not clear to me yet whether other conda dependencies might need to be reinstalled, or if applying the patch is sufficient. Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Peter Briggs [peter.bri...@manchester.ac.uk] Sent: Friday, January 05, 2018 12:06 PM To: galaxy-dev@lists.galaxyproject.org Subject: [galaxy-dev] Problems installing conda dependencies for seq2hla tool Dear Devs I've been having problems installing the jjohnson/seq2hla tool from the toolshed on our production instance (running release_17.09). Specifically Galaxy has been unable to successfully install the seq2HLA 2.2 dependency via conda. However it seems to work fine when installing into a clean local instance. Initially the problem seemed to be that our version of conda was too old (we had 4.2.13), so while the __seq2HLA@2.2 env was being created, the 'seq2hla' package couldn't be installed into it (so Galaxy thought that the requirement was satifsfied but essentially it was broken). I updated conda to 4.3.24 and checked that our "ensure_conda_channels" setting matched the one in 'galaxy.ini.sample'. Now when I try to reinstall the seq2hla tool, the dependency cannot be installed at all - there seems to be an issue with the R 3.1.2-2 conda package which this needs to pull in: ERROR conda.core.link:_execute_actions(337): An error occurred while installing package 'defaults::r-3.1.2-2'. PaddingError: Placeholder of length '80' too short in package /X/galaxy/production/tool_dependencies/_conda/envs/__seq2HLA@2.2/bin/Rscript. The package must be rebuilt with conda-build > 2.0. ... above is the path to the Galaxy installation directory, and the full path does exceed 80 characters. Presumably this worked okay on the local Galaxy install because there the path is shorter than the 80 character limit. Is there any workaround or fix that I can use to get this working on our production instance? Also, is it generally safe to update the Galaxy conda installation as I have done? Is there a recommended conda version for Galaxy? Thanks for your help, best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Problems installing conda dependencies for seq2hla tool
Dear Devs I've been having problems installing the jjohnson/seq2hla tool from the toolshed on our production instance (running release_17.09). Specifically Galaxy has been unable to successfully install the seq2HLA 2.2 dependency via conda. However it seems to work fine when installing into a clean local instance. Initially the problem seemed to be that our version of conda was too old (we had 4.2.13), so while the __seq2HLA@2.2 env was being created, the 'seq2hla' package couldn't be installed into it (so Galaxy thought that the requirement was satifsfied but essentially it was broken). I updated conda to 4.3.24 and checked that our "ensure_conda_channels" setting matched the one in 'galaxy.ini.sample'. Now when I try to reinstall the seq2hla tool, the dependency cannot be installed at all - there seems to be an issue with the R 3.1.2-2 conda package which this needs to pull in: ERROR conda.core.link:_execute_actions(337): An error occurred while installing package 'defaults::r-3.1.2-2'. PaddingError: Placeholder of length '80' too short in package /X/galaxy/production/tool_dependencies/_conda/envs/__seq2HLA@2.2/bin/Rscript. The package must be rebuilt with conda-build > 2.0. ... above is the path to the Galaxy installation directory, and the full path does exceed 80 characters. Presumably this worked okay on the local Galaxy install because there the path is shorter than the 80 character limit. Is there any workaround or fix that I can use to get this working on our production instance? Also, is it generally safe to update the Galaxy conda installation as I have done? Is there a recommended conda version for Galaxy? Thanks for your help, best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] tool publishing
Hello Matthias
My reading of the planemo docs it was that "shed_create" just made the
repository, and that it had to be followed by the "shed_update" command in
order to upload the actual content. (The documentation could probably be
clearer on this I think.)
I'm not sure about the error from "shed_diff", maybe related to the repository
not having any content.
Best wishes
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of
Matthias Bernt [m.be...@ufz.de]
Sent: Monday, December 18, 2017 9:37 AM
To: Galaxy Dev List
Subject: [galaxy-dev] tool publishing
Dear dev-list,
I'm just trying to publish my first tool to the testtoolshed via
planemo. I got some problem during the process. Maybe someone can
kickstart me (I guess I just forgot something).
I followed http://planemo.readthedocs.io/en/latest/publishing.html
The step
`planemo shed_create --shed_target testtoolshed`
```
Repository created
cd '/home/berntm/projects/mb-galaxy-tools/fasta_regex_finder' && git
rev-parse HEAD
cd '/home/berntm/projects/mb-galaxy-tools/fasta_regex_finder' && git
diff --quiet
Repository [fasta_regex_finder] does not exist in the targeted Tool Shed.
```
When I check online the repository is there but no changeset seems
available. Retrying fails (expected):
`planemo shed_create --shed_target testtoolshed`
```
Unexpected HTTP status code: 400: {"err_msg": "You already own a
repository named fasta_regex_finder<\/b>, please choose a different
name.", "err_code": 48}
Also shed_diff fails
`fasta_regex_finder git:(master) ✗ planemo shed_diff --shed_target
testtoolshed`
```
shed_diff: Repository [fasta_regex_finder] does not exist in the
targeted Tool Shed.
```
Any idea?
Cheers,
Matthias
--
---
Matthias Bernt
Bioinformatics Service
Molekulare Systembiologie (MOLSYB)
Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
Helmholtz Centre for Environmental Research GmbH - UFZ
Permoserstraße 15, 04318 Leipzig, Germany
Phone +49 341 235 482296,
m.be...@ufz.de, www.ufz.de
Sitz der Gesellschaft/Registered Office: Leipzig
Registergericht/Registration Office: Amtsgericht Leipzig
Handelsregister Nr./Trade Register Nr.: B 4703
Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
MinDirig Wilfried Kraus
Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
Prof. Dr. Dr. h.c. Georg Teutsch
Administrative Geschäftsführerin/ Administrative Managing Director:
Prof. Dr. Heike Graßmann
---
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
Re: [galaxy-dev] Enable viewing of large HTML files
Hello Martin Thanks for the quick reply and for the pointer to the appropriate code, that's great - I'll take a look at tweaking the limit in our local Galaxy if it continues to be a problem. Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Martin Čech [mar...@bx.psu.edu] Sent: Friday, November 03, 2017 2:47 PM To: Peter Briggs Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Enable viewing of large HTML files Hi Peter, I believe the 1MB is hard coded in https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/managers/hdas.py#L222 and cannot be configured. That said you can change it on your instance, I have also opened an issue to allow configuring this value. https://github.com/galaxyproject/galaxy/issues/4924 Martin On Fri, Nov 3, 2017 at 8:00 AM Peter Briggs <peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>> wrote: Dear Devs We have a locally developed tool which takes in a collection of Fastq pairs and produces a number of outputs which includes a number of HTML files. For relatively small numbers of input files all the HTML outputs can be viewed without issue in Galaxy. However we've observed that for large collections of Fastqs, the size of one of the HTML files can be very large (several megabytes) and by default attempting to view these in Galaxy results in the "This dataset is large and only the first megabyte is shown below" message, followed by the initial HTML markup rendered as text. As clicking on the "show all" link renders the entire HTML file correctly, is there a way to force Galaxy to shortcut the truncation for HTML (or at least apply a larger limit for determining when files are too large to be shown all at once)? We're currently using 17.05 in our production instance. Thanks for advance for any help or suggestions, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk> Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Enable viewing of large HTML files
Dear Devs We have a locally developed tool which takes in a collection of Fastq pairs and produces a number of outputs which includes a number of HTML files. For relatively small numbers of input files all the HTML outputs can be viewed without issue in Galaxy. However we've observed that for large collections of Fastqs, the size of one of the HTML files can be very large (several megabytes) and by default attempting to view these in Galaxy results in the "This dataset is large and only the first megabyte is shown below" message, followed by the initial HTML markup rendered as text. As clicking on the "show all" link renders the entire HTML file correctly, is there a way to force Galaxy to shortcut the truncation for HTML (or at least apply a larger limit for determining when files are too large to be shown all at once)? We're currently using 17.05 in our production instance. Thanks for advance for any help or suggestions, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Conda dependency install problems for DESeq2 tool
Hello Dave Thanks for the quick response, that seems to have fixed it - the r-getopt conda package installs without a problem and the readline.so.6/missing symbol error has gone away. (DESeq2 is failing with an R error now, but I suspect that's a separate problem.) Thanks again, Best wishes Peter On 23/08/17 15:28, Dave Bouvier wrote: Peter, We recently resolved a similar issue on usegalaxy.org, part of which involved rebuilding some of deseq2's dependencies for R version 3.3.1, which is the currently preferred version for bioconda R packages. You should be able to get the right versions and builds of deseq2 and its dependencies with the following command: conda_/bin/conda create --name __bioconductor-deseq2@1.14.1 --override-channels -c iuc -c bioconda -c conda-forge -c defaults -c r bioconductor-deseq2=1.14.1 r-base=3.3.1 This will force installation of R 3.3.1 bioconda builds of r-getopt and other dependencies, which were rebuilt with conda-build > 2.0 a few months ago. - Dave Bouvier http://galaxyproject.org http://usegalaxy.org On 08/23/2017 08:38 AM, Peter Briggs wrote: Dear devs I'm experiencing problems with the conda dependencies for the DESeq2 tool (https://toolshed.g2.bx.psu.edu/repository?repository_id=1f158f7565dc70f9) installed into our production instance. When this tool is run the only output is: /PATH/TO/job_working_directory/025/25029/conda-env/lib/R/bin/exec/R: symbol lookup error: /PATH/TO/job_working_directory/025/25029/conda-env/lib/R/bin /exec/../../lib/../../libreadline.so.6: undefined symbol: PC I've tried manually reinstalling the two dependencies (bioconductor-deseq2@1.14.2 and r-getopt@1.20.0) by moving the relevant directories from 'tool_dependencies/_conda/envs' and then doing e.g. conda_/bin/conda create -n __r-getopt@1.20.0 --override-channels -c iuc -c bioconda -c conda-forge -c defaults -c r r-getopt=1.20.0 to see if this fixes the problem. However while it completes okay for the bioconductor-deseq2 package, for r-getopt I get the following failure: PaddingError: Placeholder of length '80' too short in package r::r-base-3.2.2-0. The package must be rebuilt with conda-build > 2.0. I've tried doing 'conda_/bin/conda clean -pt' and retrying but I get the same error. Any help to fix this is greatly appreciated, many thanks Best wishes Peter Ps conda version is 4.2.13 and Galaxy is release_17.05; I'm trying to rebuild the conda environments manually because I couldn't see anything within the Galaxy UI which would let me do it there. -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Conda dependency install problems for DESeq2 tool
Dear devs I'm experiencing problems with the conda dependencies for the DESeq2 tool (https://toolshed.g2.bx.psu.edu/repository?repository_id=1f158f7565dc70f9) installed into our production instance. When this tool is run the only output is: /PATH/TO/job_working_directory/025/25029/conda-env/lib/R/bin/exec/R: symbol lookup error: /PATH/TO/job_working_directory/025/25029/conda-env/lib/R/bin /exec/../../lib/../../libreadline.so.6: undefined symbol: PC I've tried manually reinstalling the two dependencies (bioconductor-deseq2@1.14.2 and r-getopt@1.20.0) by moving the relevant directories from 'tool_dependencies/_conda/envs' and then doing e.g. conda_/bin/conda create -n __r-getopt@1.20.0 --override-channels -c iuc -c bioconda -c conda-forge -c defaults -c r r-getopt=1.20.0 to see if this fixes the problem. However while it completes okay for the bioconductor-deseq2 package, for r-getopt I get the following failure: PaddingError: Placeholder of length '80' too short in package r::r-base-3.2.2-0. The package must be rebuilt with conda-build > 2.0. I've tried doing 'conda_/bin/conda clean -pt' and retrying but I get the same error. Any help to fix this is greatly appreciated, many thanks Best wishes Peter Ps conda version is 4.2.13 and Galaxy is release_17.05; I'm trying to rebuild the conda environments manually because I couldn't see anything within the Galaxy UI which would let me do it there. -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Broken image rendering in tool XML help when serving from subdirectory
Dear devs The rendering of images in tool XML 'help' sections is broken for release_17.05 when serving Galaxy from a subdirectory using proxy-prefix. The tags are rendered in the tool helps but point to relative URLs which don't include the prefix (e.g. "admin_toolshed/static/images/756ced91f537cdf0/norm_IGVsnapshot_indFiles.png" rather than "/admin_toolshed/static/images/756ced91f537cdf0/norm_IGVsnapshot_indFiles.png"). I've looked at the Galaxy code but so far I haven't been able to see where these tags are generated (nor whether the proxy prefix is even available to the Galaxy webapp). This problem doesn't appear to be in release_17.01 (i.e. images display regardless of proxy-prefix), but it is still present in the 'dev' branch. I believe it's independent of the PATH_TO_IMAGES issue raised previously. Can anyone suggest any starting points for looking into how to fix this? Thanks for your help - I really like to get this fixed as it's breaking the very useful inline documentation for e.g. 'deepTools', as well for a locally developed tool we're currently trying to roll out on our production instance. Best wishes Peter On 10/08/17 16:12, John Chilton wrote: There is an open issue for these paths not working with vanilla Galaxy tools https://github.com/galaxyproject/galaxy/issues/1676. -John On Wed, Aug 9, 2017 at 10:03 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote: I'm puzzled now too, cross reference https://github.com/galaxy-iuc/standards/pull/46 and the original: https://github.com/galaxy-iuc/standards/issues/12 Peter On Wed, Aug 9, 2017 at 3:01 PM, Peter Briggs <peter.bri...@manchester.ac.uk> wrote: Hello Peter Thanks for the fast response. From reading around elsewhere I'd understood that using $PATH_TO_IMAGES was now deprecated? I think I tried it anyway and it didn't seem to help. But I might have missed something, so I'd be interested to know if it works for you. Best wishes Peter On 09/08/17 14:57, Peter Cock wrote: It looks like something important is not showing in the best practise website, special variable $PATH_TO_IMAGES as per the linked example: .. image:: $PATH_TO_IMAGES/slop-glyph.png Let me look into this, Peter On Wed, Aug 9, 2017 at 2:52 PM, Peter Briggs <peter.bri...@manchester.ac.uk> wrote: Dear devs I've been updating the section for a locally developed tool and want to include some images, but I've had some issues trying to see them when testing locally. I've followed the instructions outlined here: https://github.com/galaxy-iuc/standards/blob/master/docs/best_practices/tool_xml.rst#including-images (i.e. put the images files into a 'static/images/' subdir of the tool directory and reference them using ".. image:: picture.png") However when I load the tool into a Galaxy using "planemo serve ..." the images fail to render: I just get a line of text with the image filename. The same thing happens when I add the tool manually on our local server (i.e. by adding a reference to the tool XML in the appropriate conf.xml file). If I upload the same tool files to a toolshed and then install from there, I do see the images rendered correctly. I'm a bit baffled as to why there should be a difference. Is there an option or setting I've missed to turn on the image rendering for non-shed-installed tools? Or have I made some other mistake? Thanks for your help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] htseq-count/deseq2 conda dependency issues
Hello Marius Thanks for the prompt reply & explanation of the channel ordering. Checking galaxy.ini, the ordering isn't explicitly set for our instance. I'm guessing that in this case the ordering is taken from whatever Galaxy's default ordering is for that version. Is it better to explicitly set the ordering in future? Also thanks for the tip-off re updating conda, doing that now. Thanks for your help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Marius van den Beek [m.vandenb...@gmail.com] Sent: Thursday, August 17, 2017 5:47 PM To: Peter Briggs Cc: Galaxy Dev List Subject: Re: [galaxy-dev] htseq-count/deseq2 conda dependency issues Hi Peter, indeed the defaults channel should now be the lowest priority of all conda channels. This recommended setting in galaxy.ini is now conda_ensure_channels = iuc,bioconda,conda-forge,defaults,r Unfortunately these changes appear as changes are made within bioconda, which usually do not coincide with galaxy releases. That means that these settings may need to be adjusted occasionally. On a slightly unrelated note, if you have upgraded to galaxy 17.05 you should also upgrade conda to 4.2.13 if you haven't done so (conda install conda=4.2.13), for best experience. Hope that helps, Marius On 17 August 2017 at 17:12, Peter Briggs <peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>> wrote: Dear devs I've recently encountered some strange issues when running the htseq-count and deseq2 tools installed from the toolshed onto our production instance. In both cases there have been problems with a couple of the conda dependency installations. Specifically: - For htseq-count, runs failed with an error message about a missing libbz2.so.1.0. - For deseq2, runs failed silently first with the same "missing libbz2.so.1.0" message, then when this was fixed, with a new message about an undefined symbol in libreadline.so.6. In both cases I seemed to be able to fix the problems manually by activating the appropriate conda environments (under .../tool_dependencies/_conda/envs) and reinstalling the affected packages from different sources: - htseq-count: in __samtools@1.3.1, reinstalled bzip2 1.0.2 from 'conda-forge' rather than 'defaults' - deseq2: in __r-getopt@1.20.0, did the same for bzip2 and also got readline 6.3 from 'conda-forge' rather than 6.2 from 'defaults' However: deseq2 installed into a local Galaxy instance doesn't seem to have this problem, so I'm guessing it's something peculiar to the setup of the production instance. Can anyone suggest what might be happening here (e.g. some issue with the conda config for the production instance?), and how it might be avoided in future? Thanks for your help, Best wishes Peter (PS if it helps: the production instance is running Scientific Linux 6.8, and has recently been updated from v16.07 to v17.05. Both the tools were installed prior to the upgrade.) -- Peter Briggs peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk> Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] htseq-count/deseq2 conda dependency issues
Dear devs I've recently encountered some strange issues when running the htseq-count and deseq2 tools installed from the toolshed onto our production instance. In both cases there have been problems with a couple of the conda dependency installations. Specifically: - For htseq-count, runs failed with an error message about a missing libbz2.so.1.0. - For deseq2, runs failed silently first with the same "missing libbz2.so.1.0" message, then when this was fixed, with a new message about an undefined symbol in libreadline.so.6. In both cases I seemed to be able to fix the problems manually by activating the appropriate conda environments (under .../tool_dependencies/_conda/envs) and reinstalling the affected packages from different sources: - htseq-count: in __samtools@1.3.1, reinstalled bzip2 1.0.2 from 'conda-forge' rather than 'defaults' - deseq2: in __r-getopt@1.20.0, did the same for bzip2 and also got readline 6.3 from 'conda-forge' rather than 6.2 from 'defaults' However: deseq2 installed into a local Galaxy instance doesn't seem to have this problem, so I'm guessing it's something peculiar to the setup of the production instance. Can anyone suggest what might be happening here (e.g. some issue with the conda config for the production instance?), and how it might be avoided in future? Thanks for your help, Best wishes Peter (PS if it helps: the production instance is running Scientific Linux 6.8, and has recently been updated from v16.07 to v17.05. Both the tools were installed prior to the upgrade.) -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Image in tool XML "help" section only renders when installed from toolshed?
Hello Peter Thanks for the fast response. From reading around elsewhere I'd understood that using $PATH_TO_IMAGES was now deprecated? I think I tried it anyway and it didn't seem to help. But I might have missed something, so I'd be interested to know if it works for you. Best wishes Peter On 09/08/17 14:57, Peter Cock wrote: It looks like something important is not showing in the best practise website, special variable $PATH_TO_IMAGES as per the linked example: .. image:: $PATH_TO_IMAGES/slop-glyph.png Let me look into this, Peter On Wed, Aug 9, 2017 at 2:52 PM, Peter Briggs <peter.bri...@manchester.ac.uk> wrote: Dear devs I've been updating the section for a locally developed tool and want to include some images, but I've had some issues trying to see them when testing locally. I've followed the instructions outlined here: https://github.com/galaxy-iuc/standards/blob/master/docs/best_practices/tool_xml.rst#including-images (i.e. put the images files into a 'static/images/' subdir of the tool directory and reference them using ".. image:: picture.png") However when I load the tool into a Galaxy using "planemo serve ..." the images fail to render: I just get a line of text with the image filename. The same thing happens when I add the tool manually on our local server (i.e. by adding a reference to the tool XML in the appropriate conf.xml file). If I upload the same tool files to a toolshed and then install from there, I do see the images rendered correctly. I'm a bit baffled as to why there should be a difference. Is there an option or setting I've missed to turn on the image rendering for non-shed-installed tools? Or have I made some other mistake? Thanks for your help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
[galaxy-dev] Image in tool XML "help" section only renders when installed from toolshed?
Dear devs I've been updating the section for a locally developed tool and want to include some images, but I've had some issues trying to see them when testing locally. I've followed the instructions outlined here: https://github.com/galaxy-iuc/standards/blob/master/docs/best_practices/tool_xml.rst#including-images (i.e. put the images files into a 'static/images/' subdir of the tool directory and reference them using ".. image:: picture.png") However when I load the tool into a Galaxy using "planemo serve ..." the images fail to render: I just get a line of text with the image filename. The same thing happens when I add the tool manually on our local server (i.e. by adding a reference to the tool XML in the appropriate conf.xml file). If I upload the same tool files to a toolshed and then install from there, I do see the images rendered correctly. I'm a bit baffled as to why there should be a difference. Is there an option or setting I've missed to turn on the image rendering for non-shed-installed tools? Or have I made some other mistake? Thanks for your help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Galaxy Reports Webapp Login
Hello Timo For our production setups I've used the htpasswd mechanism as a simple way to protect the reports from unauthorised access. The details for doing this using nginx are part of the write up here: http://galacticengineer.blogspot.co.uk/2015/06/exposing-galaxy-reports-via-nginx-in.html HTH Best wishes Peter On 27/03/17 12:29, Timo Janßen wrote: Hi, is it possible to implement some kind of login for the usage reports webapp, so that only admins can see the website? In our current setup the tool is running on a server with many other users so that anyone who knows the port can open the website and see potentially sensitive data. Best regards, Timo ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Relabeling dataset pairs in 'list:paired' collection
Hello John Thanks for your extensive and detailed reply, and also for making the new tool. This approach (implementing the operation within the framework) looks vastly better than my hack for generating a new collection, and should address the current use case that I have. (I agree also, it makes sense that collection operations should be as generic as possible.) I'll add any comments to the #3603 PR, however between you and Marius I think so far everything I'd thought of (and more) is already covered there. Thanks again for looking into this! Best wishes Peter On 13/02/17 19:43, John Chilton wrote: Thanks for your interest in this topic. The collection operations exist the way they do as tools distributed with the core framework because they can't be expressed as normal tools and they utilize abstractions that I don't consider public at this time (or really have any confidence in making public). For this reason I think it would be best to just implement a collection operation inside of the core framework - Philip Mabon created such an operation as part of this https://github.com/galaxyproject/galaxy/pull/2771 and it was merged. I think the kinds of things that would benefit from collection operations tend to be expressible in fairly generic terms so they don't apply to just one field. I think your relabelling example is exactly such a case. I have opened a PR with the start of such as tool that I hope would address your use case with https://github.com/galaxyproject/galaxy/pull/3603. I'm calling it relabelling from a file because I'd also like to implement a relabelling with a function someday but there has been some push back on extending these collection operations to consume small functions. Even if that were implemented though - the use case of reading from a file is pretty different and frankly probably more practical for typical workflows that might make use of these. Perhaps we can continue this conversation on the #3603. Thanks again, -John On Mon, Feb 13, 2017 at 11:11 AM, Peter Briggs <peter.bri...@manchester.ac.uk> wrote: Dear Developers Is there an existing tool or mechanism that can be used to duplicate a "list of pairs" dataset collection, keeping the paired datasets the same but relabeling each pair with a new identifier taken from a user supplied file or list? I've cobbled together my own tool to try and do something like this: https://github.com/pjbriggs/Amplicon_analysis-galaxy/blob/77340d8bb2470a646deba4933625413fc70985d1/relabel_samples.xml and while it works, it doesn't feel like a good solution as it creates duplicates of the datasets from the first collection and consumes additional disk/quota space unnecessarily. (This is particularly undesirable as we expect that the input collections might be relatively large numbers of FASTQ pairs e.g. 30 or more.) Looking at some of the 'Collection Operations' tools that come with Galaxy, it appears that these are able to create new collections without making duplicate datasets, which seems much better. But these tools work by directly invoking Python classes from the Galaxy core, so I don't know if a similar approach could be used in a non-core tool. Any advice or suggestions are very welcome! Thanks Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Relabeling dataset pairs in 'list:paired' collection
Dear Developers Is there an existing tool or mechanism that can be used to duplicate a "list of pairs" dataset collection, keeping the paired datasets the same but relabeling each pair with a new identifier taken from a user supplied file or list? I've cobbled together my own tool to try and do something like this: https://github.com/pjbriggs/Amplicon_analysis-galaxy/blob/77340d8bb2470a646deba4933625413fc70985d1/relabel_samples.xml and while it works, it doesn't feel like a good solution as it creates duplicates of the datasets from the first collection and consumes additional disk/quota space unnecessarily. (This is particularly undesirable as we expect that the input collections might be relatively large numbers of FASTQ pairs e.g. 30 or more.) Looking at some of the 'Collection Operations' tools that come with Galaxy, it appears that these are able to create new collections without making duplicate datasets, which seems much better. But these tools work by directly invoking Python classes from the Galaxy core, so I don't know if a similar approach could be used in a non-core tool. Any advice or suggestions are very welcome! Thanks Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message
Hello Dan Thanks for merging in the changes, just to let you know that these packages seem to install okay for me now from the toolshed - much appreciated! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Daniel Blankenberg [d...@bx.psu.edu] Sent: Friday, February 03, 2017 2:46 PM To: Jennifer Hsieh Cc: Nikolaos Tur; Galaxy Dev List; Peter Briggs Subject: Re: [galaxy-dev] Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message Hi all, These packages should now install correctly from the toolshed with Peter’s PR. Please let me know if you encounter additional issues. Thanks for using Galaxy, Dan On Jan 31, 2017, at 10:47 PM, Jennifer Hsieh <humbleambit...@gmail.com<mailto:humbleambit...@gmail.com>> wrote: Thank so much! This is exactly what I needed. What a gem :) Best, Jen On Tue, Jan 31, 2017 at 3:48 PM, Nikolaos Tur <nikolaos@gmail.com<mailto:nikolaos@gmail.com>> wrote: Hi, sorry for long delay answer. Solution which Peter suggested is absolutely right - owner of packages has to add changes. Because I did not have any reaction to my message but needed quick solution and wanted to stay in consistent with standard Galaxy installation procedure and neither do any fork of Daniel's packages and setup then local toolsched repository for tools or install these packages locally, I did some trick. First I installed these packages. Installation of course failed. Then I open them in web interface, uninstalled them, modified tool_dependencies.xml files (see below) and installed them again from the web interface. Sorry if I miss something that was several months ago. shell_command was modified. ... hg clone -r ba0341816b15fff5e0a554658d6579a60e1b12b1 https://bitbucket.org/dan/pybamtools pyBamTools $INSTALL_DIR/lib/python export PYTHONPATH=$PYTHONPATH:$INSTALL_DIR/lib/python && sed -i '/use_setuptools()/ s/^/#/' setup.py && sed -i '/from distribute_setup import use_set uptools/ s/^/#/' setup.py && python setup.py install --home $INSTALL_DIR $INSTALL_DIR/lib/python ... instead of export PYTHONPATH=$PYTHONPATH:$INSTALL_DIR/lib/python && python setup.py install --home $INSTALL_DIR Idea I think is clear, after clone of tools, modify first setup.py files and then compile and install. Hope it can be useful, Nikolaos. On Tue, Jan 10, 2017 at 12:54 PM, Jennifer Hsieh <humbleambit...@gmail.com<mailto:humbleambit...@gmail.com>> wrote: Hi Peter, Thank you so much. I'll see what I can figure out with these pull requests. I'm still quite inexperienced with these types of files. This is probably a good opportunity to learn some new code/structures. If I can't manage to figure it out, I hope there's another variant caller with different dependencies I can use for my project. Best, Jen On 9 Jan 2017 4:06 p.m., "Peter Briggs" <peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>> wrote: Hello Jennifer I came across the same problem with the naive variant caller at the end of last year, when trying to upgrade our local Galaxy instance. As you describe, the issue is actually with the two dependency packages: -- package_pybamparser_0_0_1 -- package_pybamtools_0_0_2 I think I found the same solution as Nikolaos to the underlying problem (i.e. the setup.py files in these two packages are no longer compatible with the newer versions of the Python 'setuptools'), but I didn't have time to follow it up then. I've now submitted pull requests with my attempted fixes on the Bitbucket repositories which hold the source code for the dependencies (URLs below), so at least you can see how I've tried to address the problem. However I don't know of an easy way to apply these changes inside the Galaxy context - my hope is that the tool maintainer (Daniel Blankenberg, cc'ed) will take a look and update the dependencies on the toolshed. Not sure if this helps - maybe someone else on this list has a way to apply the patches directly? Best wishes Peter Pull request URLs: -- https://bitbucket.org/dan/pybamtools/pull-requests/1/ -- https://bitbucket.org/dan/pybamparser/pull-requests/3/ On 06/01/17 14:50, Jennifer Hsieh wrote: Hi, I've installed a local instance of Galaxy (on Ubuntu 16.04, python 2.7.12) and have run in to this exact problem (posted by Nikolaos Tur https://www.mail-archive.com/galaxy-dev@lists.galaxyproject.org/msg03955.html) while trying to add naive_variant_caller tool. Naive_variant_caller can't install because these two dependency packages can't be installed. I basically get exactly the same error message that ends with "IOError: Could not build the egg." Unfortunately I'm very new to both Galaxy and Python (Taking Coursera Galaxy course a
Re: [galaxy-dev] Problem with 'galaxy_packages' dependency resolver in versionless mode
Hello Bjoern Yes it looks like making the 'default' directory into a symlink makes this work. Thanks for your help and sorry not to get back to you sooner! Best wishes Peter On 20/01/17 16:56, Björn Grüning wrote: Hi Peter, can you try to make /path/to/local_tool_dependencies/my_package/default a symlink? Ciao, Bjoern Am 20.01.2017 um 16:41 schrieb Peter Briggs: Dear developers I'm having problems trying to use the 'galaxy_packages' dependency resolver in versionless mode, as described in the docs at: https://docs.galaxyproject.org/en/latest/admin/dependency_resolvers.html#galaxy-packages-dependency-resolver As I understand this: if I manually create a directory structure of the form e.g. /path/to/local_tool_dependencies/my_package/default/ which contains an env.sh file, and if my dependency_resolvers_conf.xml file includes the line: then at run time Galaxy should be able to resolve the following requirement in a tool XML file: my_package Unfortunately although the "versioned" galaxy_packages resolver works as expected (when placed ahead of the "versionless" resolver), I don't seem able to get this working with Galaxy v16.10. Can anyone point out what I'm doing wrong? Thanks! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Problem with 'galaxy_packages' dependency resolver in versionless mode
Dear developers I'm having problems trying to use the 'galaxy_packages' dependency resolver in versionless mode, as described in the docs at: https://docs.galaxyproject.org/en/latest/admin/dependency_resolvers.html#galaxy-packages-dependency-resolver As I understand this: if I manually create a directory structure of the form e.g. /path/to/local_tool_dependencies/my_package/default/ which contains an env.sh file, and if my dependency_resolvers_conf.xml file includes the line: then at run time Galaxy should be able to resolve the following requirement in a tool XML file: my_package Unfortunately although the "versioned" galaxy_packages resolver works as expected (when placed ahead of the "versionless" resolver), I don't seem able to get this working with Galaxy v16.10. Can anyone point out what I'm doing wrong? Thanks! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Making symlinks in "command" section of tool XML
Hello John Thanks for the prompt reply, and the heads-up re the deprecation of the "interpreter" attribute. It looks removing this attribute from the "command" tag fixes the linking problem, as you say. I'll try the new idiom referencing $__tool_directory__ and let you know if I have any issues. Thanks for your help! Best wishes Peter On 19/01/17 15:38, John Chilton wrote: At first glace I think the problem is that when you use the ``interpreter`` tag Galaxy has really broken logic for modifying the command line to use that. ln works just like cp and mv. Those too would be broken for this tool because of the interpreter attribute I think. For this reason ``interpreter`` has been deprecated (https://docs.galaxyproject.org/en/latest/dev/schema.html#id3) and should not be used. I would drop the interpreter="bash" stuff and just where ever your script (lets say script_name.bash) appears in the command replace the reference to script_name.bash with: bash $__tool_directory__/script_name.bash I think this will work a lot better for you. Let me know if it does not. -John On Thu, Jan 19, 2017 at 10:05 AM, Peter Briggs <peter.bri...@manchester.ac.uk> wrote: Dear Developers I'm currently looking at develop a tool for a local pipeline, and want to make symlinks as part of the script in the "command" section of the tool XML - something like e.g.
[galaxy-dev] Making symlinks in "command" section of tool XML
Dear Developers I'm currently looking at develop a tool for a local pipeline, and want to make symlinks as part of the script in the "command" section of the tool XML - something like e.g.
Re: [galaxy-dev] Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message
Hello Jennifer I came across the same problem with the naive variant caller at the end of last year, when trying to upgrade our local Galaxy instance. As you describe, the issue is actually with the two dependency packages: -- package_pybamparser_0_0_1 -- package_pybamtools_0_0_2 I think I found the same solution as Nikolaos to the underlying problem (i.e. the setup.py files in these two packages are no longer compatible with the newer versions of the Python 'setuptools'), but I didn't have time to follow it up then. I've now submitted pull requests with my attempted fixes on the Bitbucket repositories which hold the source code for the dependencies (URLs below), so at least you can see how I've tried to address the problem. However I don't know of an easy way to apply these changes inside the Galaxy context - my hope is that the tool maintainer (Daniel Blankenberg, cc'ed) will take a look and update the dependencies on the toolshed. Not sure if this helps - maybe someone else on this list has a way to apply the patches directly? Best wishes Peter Pull request URLs: -- https://bitbucket.org/dan/pybamtools/pull-requests/1/ -- https://bitbucket.org/dan/pybamparser/pull-requests/3/ On 06/01/17 14:50, Jennifer Hsieh wrote: Hi, I've installed a local instance of Galaxy (on Ubuntu 16.04, python 2.7.12) and have run in to this exact problem (posted by Nikolaos Tur https://www.mail-archive.com/galaxy-dev@lists.galaxyproject.org/msg03955.html) while trying to add naive_variant_caller tool. Naive_variant_caller can't install because these two dependency packages can't be installed. I basically get exactly the same error message that ends with "IOError: Could not build the egg." Unfortunately I'm very new to both Galaxy and Python (Taking Coursera Galaxy course at the moment) and I don't really understand the solution Nikolaos described about modifying the setup.py to use "setuptools" instead of "distribute". I don't even know where to find setup.py... Can anyone help me with step-by-step details as to how to solve this dependency problem? Much Thanks! -- Jennifer Hsieh PhD Neuroscience (student) Department of Psychiatry and Mental Health University of Cape Town Tell: +27 21 447 3157 Cell: +27 74 841 4761 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tools pick up Python modules from Galaxy core rather than installed tool dependencies
Hello Nate Thanks for the follow up, and the tip off about manipulating the env variables in job_conf.xml for specific job destinations - I didn't know about that but I could imagine other situations where it might be useful! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Nate Coraor [n...@bx.psu.edu] Sent: Monday, December 19, 2016 5:50 PM To: Peter Briggs Cc: Galaxy Dev List Subject: Re: [galaxy-dev] Tools pick up Python modules from Galaxy core rather than installed tool dependencies Hi Peter, Thanks for posting your findings. You can also directly manipulate the PYTHONPATH in tags in job_conf.xml s, which may be enough, although this would be tedious for every dependency. --nate On Fri, Dec 16, 2016 at 5:35 AM, Peter Briggs <peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>> wrote: I think I've figured this out in the end, posting my solution in case anyone else comes across it in future. The issue was due to a change in the way that Python setuptools adds its installed modules to the search path (sys.path) via easy-install.pth files. Later versions of setuptools (>=25.0.0) append to rather than rewrite sys.path, so modules installed as eggs don't take precedence over older versions in the Galaxy core (for example "galaxy-ops" and "galaxy_sequence_utils"). One fix is to set the SETUPTOOLS_SYS_PATH_TECHNIQUE environment variable to "rewrite" and reinstall the modules (to update the module's "easy-install.pth" file). This should work for all recent versions of setuptools (and fixed the problems below for me), but might not work for future versions of setuptools - see http://setuptools.readthedocs.io/en/latest/history.html#v25-0-0. Alternatively: the "galaxy_sequence_utils" package (used by "fastq_paired_end_joiner", maybe others?) is available in bioconda, so using that is probably a better long-term fix (not tested yet though). But I don't think "galaxy-ops" (used by "join" and other tools) is there yet. Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk> Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org<mailto:galaxy-dev-boun...@lists.galaxyproject.org>] on behalf of Peter Briggs [peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>] Sent: Wednesday, December 14, 2016 4:13 PM To: Galaxy Dev List Subject: [galaxy-dev] Tools pick up Python modules from Galaxy core rather than installed tool dependencies Dear devs I was wondering if anyone had any suggestions as to the cause of a peculiar error I'm seeing when running the latest versions of the devteam "join" and "fastq_paired_end_joiner". The tools appear to be picking up Python modules from the Galaxy library, rather than from the installed dependencies. The PYTHONPATH appears to be correct (i.e. tool dependency appears before Galaxy library path) but sys.path shows the dependency's Python egg appearing after the Galaxy library path. E.g. for the paired-end joiner: PYTHONPATH=/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python:/mnt/rvmi/centaurus/galaxy/devel/galaxy-dist/lib sys.path = ['', '/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python', '/mnt/rvmi/centaurus/galaxy/devel/galaxy-dist/lib', '/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python/galaxy_sequence_utils-1.0.1-py2.7.egg', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python27.zip', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/plat-linux2', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-tk', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-old', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-dynload', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7/plat-linux2', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7/lib-tk', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/site-packages'] Error: Traceback (most recent call last): File "/mnt/rvmi/centaurus/galaxy/devel/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_paired_end_joiner/6a7f5d
Re: [galaxy-dev] Installation of packages failing
Thanks for the update Nate, glad you were able to fix it! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Nate Coraor [n...@bx.psu.edu] Sent: Monday, December 19, 2016 5:08 PM To: Peter Briggs Cc: Gildas Le Corguillé; James Mullan; Galaxy Dev List Subject: Re: [galaxy-dev] Installation of packages failing Hi Peter, Gildas, This was due to a power supply failure which we've fixed this morning, so this should all be working now. Thanks, --nate On Mon, Dec 19, 2016 at 5:46 AM, Peter Briggs <peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>> wrote: Hello Gildas I think the problem is more that the actual server isn't serving the packages that are already hosted there. For example: $ wget http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz ... ... --2016-12-19 10:20:47-- (try:20) http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz Connecting to depot.galaxyproject.org<http://depot.galaxyproject.org> (depot.galaxyproject.org<http://depot.galaxyproject.org>)|128.118.250.14|:80... failed: Connection timed out. Giving up. I tried this from a few different locations last week with the same result. Best wishes Peter On 19/12/16 10:08, Gildas Le Corguillé wrote: You can propose some updates (ie: URL) in this GitHub repo : https://github.com/galaxyproject/cargo-port Cheers Gildas - Gildas Le Corguillé - Bioinformatician/Bioanalyste Plateform ABiMS (Analyses and Bioinformatics for Marine Science) http://abims.sb-roscoff.fr Member of the Workflow4Metabolomics project http://workflow4metabolomics.org <http://workflow4metabolomics.org/> Station Biologique de Roscoff - UPMC/CNRS - FR2424 Place Georges Teissier 29680 Roscoff FRANCE tel: +33 2 98 29 23 81<tel:%2B33%202%2098%2029%2023%2081> ------ Le 19 déc. 2016 à 10:42, Peter Briggs <peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk> <mailto:peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk>>> a écrit : Hello James I think this might be a general problem with depot.galaxyproject.org<http://depot.galaxyproject.org> <http://depot.galaxyproject.org> serving packages, as I've been experiencing similar problems since Friday - attempts to fetch e.g. http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz also fail with timeouts. I guess it needs someone from the the Galaxy project to take a look, Best wishes Peter On 18/12/16 20:25, James Mullan wrote: Hello,* * * * I am trying to install packages with dependencies from Galaxy Main tool shed. This worked Thursday and since Friday it no longer works for me. It seems this url is no longer reachable. Examples of what I see in the log during download are shown below. Any help would be much appreciated. package_bowtie_0_12_7 Error downloading from URL http://depot.galaxyproject.org/package/linux/x86_64/bowtie/bowtie-0.12.7-linux-x86_64.tgz package_samtools_0_1_19 Exception: Error downloading from URL https://depot.galaxyproject.org/software/samtools/samtools_0.1.19_src_all.tar.bz2* * Regards James J P Mullan ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk> Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk<mailto:peter.bri...@manchester.ac.uk> Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxypro
Re: [galaxy-dev] Installation of packages failing
Hello Gildas I think the problem is more that the actual server isn't serving the packages that are already hosted there. For example: $ wget http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz ... ... --2016-12-19 10:20:47-- (try:20) http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz Connecting to depot.galaxyproject.org (depot.galaxyproject.org)|128.118.250.14|:80... failed: Connection timed out. Giving up. I tried this from a few different locations last week with the same result. Best wishes Peter On 19/12/16 10:08, Gildas Le Corguillé wrote: You can propose some updates (ie: URL) in this GitHub repo : https://github.com/galaxyproject/cargo-port Cheers Gildas - Gildas Le Corguillé - Bioinformatician/Bioanalyste Plateform ABiMS (Analyses and Bioinformatics for Marine Science) http://abims.sb-roscoff.fr Member of the Workflow4Metabolomics project http://workflow4metabolomics.org <http://workflow4metabolomics.org/> Station Biologique de Roscoff - UPMC/CNRS - FR2424 Place Georges Teissier 29680 Roscoff FRANCE tel: +33 2 98 29 23 81 -- Le 19 déc. 2016 à 10:42, Peter Briggs <peter.bri...@manchester.ac.uk <mailto:peter.bri...@manchester.ac.uk>> a écrit : Hello James I think this might be a general problem with depot.galaxyproject.org <http://depot.galaxyproject.org> serving packages, as I've been experiencing similar problems since Friday - attempts to fetch e.g. http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz also fail with timeouts. I guess it needs someone from the the Galaxy project to take a look, Best wishes Peter On 18/12/16 20:25, James Mullan wrote: Hello,* * * * I am trying to install packages with dependencies from Galaxy Main tool shed. This worked Thursday and since Friday it no longer works for me. It seems this url is no longer reachable. Examples of what I see in the log during download are shown below. Any help would be much appreciated. package_bowtie_0_12_7 Error downloading from URL http://depot.galaxyproject.org/package/linux/x86_64/bowtie/bowtie-0.12.7-linux-x86_64.tgz package_samtools_0_1_19 Exception: Error downloading from URL https://depot.galaxyproject.org/software/samtools/samtools_0.1.19_src_all.tar.bz2* * Regards James J P Mullan ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installation of packages failing
Hello James I think this might be a general problem with depot.galaxyproject.org serving packages, as I've been experiencing similar problems since Friday - attempts to fetch e.g. http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz also fail with timeouts. I guess it needs someone from the the Galaxy project to take a look, Best wishes Peter On 18/12/16 20:25, James Mullan wrote: Hello,* * * * I am trying to install packages with dependencies from Galaxy Main tool shed. This worked Thursday and since Friday it no longer works for me. It seems this url is no longer reachable. Examples of what I see in the log during download are shown below. Any help would be much appreciated. package_bowtie_0_12_7 Error downloading from URL http://depot.galaxyproject.org/package/linux/x86_64/bowtie/bowtie-0.12.7-linux-x86_64.tgz package_samtools_0_1_19 Exception: Error downloading from URL https://depot.galaxyproject.org/software/samtools/samtools_0.1.19_src_all.tar.bz2* * Regards James J P Mullan ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tools pick up Python modules from Galaxy core rather than installed tool dependencies
I think I've figured this out in the end, posting my solution in case anyone else comes across it in future. The issue was due to a change in the way that Python setuptools adds its installed modules to the search path (sys.path) via easy-install.pth files. Later versions of setuptools (>=25.0.0) append to rather than rewrite sys.path, so modules installed as eggs don't take precedence over older versions in the Galaxy core (for example "galaxy-ops" and "galaxy_sequence_utils"). One fix is to set the SETUPTOOLS_SYS_PATH_TECHNIQUE environment variable to "rewrite" and reinstall the modules (to update the module's "easy-install.pth" file). This should work for all recent versions of setuptools (and fixed the problems below for me), but might not work for future versions of setuptools - see http://setuptools.readthedocs.io/en/latest/history.html#v25-0-0. Alternatively: the "galaxy_sequence_utils" package (used by "fastq_paired_end_joiner", maybe others?) is available in bioconda, so using that is probably a better long-term fix (not tested yet though). But I don't think "galaxy-ops" (used by "join" and other tools) is there yet. Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ____ From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Peter Briggs [peter.bri...@manchester.ac.uk] Sent: Wednesday, December 14, 2016 4:13 PM To: Galaxy Dev List Subject: [galaxy-dev] Tools pick up Python modules from Galaxy core rather than installed tool dependencies Dear devs I was wondering if anyone had any suggestions as to the cause of a peculiar error I'm seeing when running the latest versions of the devteam "join" and "fastq_paired_end_joiner". The tools appear to be picking up Python modules from the Galaxy library, rather than from the installed dependencies. The PYTHONPATH appears to be correct (i.e. tool dependency appears before Galaxy library path) but sys.path shows the dependency's Python egg appearing after the Galaxy library path. E.g. for the paired-end joiner: PYTHONPATH=/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python:/mnt/rvmi/centaurus/galaxy/devel/galaxy-dist/lib sys.path = ['', '/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python', '/mnt/rvmi/centaurus/galaxy/devel/galaxy-dist/lib', '/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python/galaxy_sequence_utils-1.0.1-py2.7.egg', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python27.zip', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/plat-linux2', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-tk', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-old', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-dynload', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7/plat-linux2', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7/lib-tk', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/site-packages'] Error: Traceback (most recent call last): File "/mnt/rvmi/centaurus/galaxy/devel/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_paired_end_joiner/6a7f5da7c76d/fastq_paired_end_joiner/fastq_paired_end_joiner.py", line 158, in main() File "/mnt/rvmi/centaurus/galaxy/devel/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_paired_end_joiner/6a7f5da7c76d/fastq_paired_end_joiner/fastq_paired_end_joiner.py", line 136, in main joiner = fq.fastqJoiner( input1_type, paste=paste ) TypeError: __init__() got an unexpected keyword argument 'paste' I know it's a long shot but can anyone suggest where to start looking at where this is going wrong? Thanks, Peter Ps I'm using Galaxy v16.07 and the problem occurs on our production platform but not in a local Galaxy install on my desktop. -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at
[galaxy-dev] devteam dropped support for installing tool dependencies from toolshed?
Dear developers It looks like some updates to 'devteam'-maintained toolshed tools have dropped support for tool_dependencies.xml (e.g. fastqc, freebayes), so their dependencies cannot be automatically installed from the toolshed. Is this correct? I assume that it's part of the move to enforcing conda dependency resolution. Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Tools pick up Python modules from Galaxy core rather than installed tool dependencies
Dear devs I was wondering if anyone had any suggestions as to the cause of a peculiar error I'm seeing when running the latest versions of the devteam "join" and "fastq_paired_end_joiner". The tools appear to be picking up Python modules from the Galaxy library, rather than from the installed dependencies. The PYTHONPATH appears to be correct (i.e. tool dependency appears before Galaxy library path) but sys.path shows the dependency's Python egg appearing after the Galaxy library path. E.g. for the paired-end joiner: PYTHONPATH=/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python:/mnt/rvmi/centaurus/galaxy/devel/galaxy-dist/lib sys.path = ['', '/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python', '/mnt/rvmi/centaurus/galaxy/devel/galaxy-dist/lib', '/mnt/rvmi/centaurus/galaxy/devel/tool_dependencies/galaxy_sequence_utils/1.0.1/iuc/package_galaxy_sequence_utils_1_0_1/c1ab450748ba/lib/python/galaxy_sequence_utils-1.0.1-py2.7.egg', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python27.zip', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/plat-linux2', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-tk', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-old', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/lib-dynload', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7/plat-linux2', '/mnt/rvmi/centaurus/galaxy/python/2.7.10/lib/python2.7/lib-tk', '/home/centaurus-galaxy/fastq_paired_end_joiner_problem/venv/lib/python2.7/site-packages'] Error: Traceback (most recent call last): File "/mnt/rvmi/centaurus/galaxy/devel/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_paired_end_joiner/6a7f5da7c76d/fastq_paired_end_joiner/fastq_paired_end_joiner.py", line 158, in main() File "/mnt/rvmi/centaurus/galaxy/devel/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_paired_end_joiner/6a7f5da7c76d/fastq_paired_end_joiner/fastq_paired_end_joiner.py", line 136, in main joiner = fq.fastqJoiner( input1_type, paste=paste ) TypeError: __init__() got an unexpected keyword argument 'paste' I know it's a long shot but can anyone suggest where to start looking at where this is going wrong? Thanks, Peter Ps I'm using Galaxy v16.07 and the problem occurs on our production platform but not in a local Galaxy install on my desktop. -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Broken logo 'src' in masthead for Galaxy 16.07 served from subdirectory
Hello Saskia
Thanks, yes that seems to have fixed it. Pretty obvious really!
I wonder, is your Galaxy instance also served from a subdirectory?
Thanks again,
Best wishes
Peter
On 08/12/16 13:21, Saskia Hiltemann wrote:
hmm, I now remember having same thing when I first upgraded, I changed a
line in galaxy.masthead.mako:
--- a/templates/webapps/galaxy/galaxy.masthead.mako
+++ b/templates/webapps/galaxy/galaxy.masthead.mako
@@ -24,7 +24,7 @@
'terms_url' : app.config.get("terms_url", ""),
'allow_user_creation' : app.config.allow_user_creation,
'logo_url' : h.url_for(app.config.get(
'logo_url', '/')),
-'logo_src' : h.url_for( app.config.get(
'logo_src', '../../../static/images/galaxyIcon_noText
+'logo_src' : h.url_for( app.config.get(
'logo_src', '/static/images/galaxyIcon_noText.png' )
'is_admin_user' : trans.user_is_admin(),
'active_view' : active_view,
'ftp_upload_dir':
app.config.get("ftp_upload_dir", None),
@@ -78,4 +78,4 @@
Does this help at all?
Saskia
On 08-12-16 14:04, Peter Briggs wrote:
Dear developers
Since upgrading to release_16.07 I've noticed a broken link to the
masthead icon (i.e. the Galaxy logo 'galaxyIcon_noText.png') in our
local Galaxy installations, which are all served from subdirectories
of the webserver, e.g.
https://centaurus.itservices.manchester.ac.uk/galaxy/
The icon is visible when this URL is visited but appears to be broken
when using parts of Galaxy served from their own subdirectories e.g.
the login page (under
https://centaurus.itservices.manchester.ac.uk/galaxy/user/login?...)
or the admin interface
(https://centaurus.itservices.manchester.ac.uk/galaxy/admin).
At the top level the logo "src" is points correctly to
"/galaxy/static/images/galaxyIcon_noText.png"; in the subdirs
(users,admin etc) it points to
"../../../static/images/galaxyIcon_noText.png" (which cannot be loaded).
As far as I can work out, in 16.07 the "src" location is generated
from templates/webapps/galaxy/galaxy.masthead.mako, but takes its
value from the 'logo_src' parameter that is set in
lib/galaxy/managers/configuration.py:
...
# TODO: this doesn't seem right
'logo_url' : lambda i, k, **c: self.url_for(
i.get( k, '/' ) ),
'logo_src' : lambda i, k, **c: self.url_for(
'/static/images/galaxyIcon_noText.png' ),
...
However I have no idea how to try and fix it - presumably it needs to
incorporate the proxy-prefix setting somehow. (Btw the 'logo_url' does
seem to point to the correct location however.)
Can someone suggest a fix?
Thanks
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Broken logo 'src' in masthead for Galaxy 16.07 served from subdirectory
Dear developers Since upgrading to release_16.07 I've noticed a broken link to the masthead icon (i.e. the Galaxy logo 'galaxyIcon_noText.png') in our local Galaxy installations, which are all served from subdirectories of the webserver, e.g. https://centaurus.itservices.manchester.ac.uk/galaxy/ The icon is visible when this URL is visited but appears to be broken when using parts of Galaxy served from their own subdirectories e.g. the login page (under https://centaurus.itservices.manchester.ac.uk/galaxy/user/login?...) or the admin interface (https://centaurus.itservices.manchester.ac.uk/galaxy/admin). At the top level the logo "src" is points correctly to "/galaxy/static/images/galaxyIcon_noText.png"; in the subdirs (users,admin etc) it points to "../../../static/images/galaxyIcon_noText.png" (which cannot be loaded). As far as I can work out, in 16.07 the "src" location is generated from templates/webapps/galaxy/galaxy.masthead.mako, but takes its value from the 'logo_src' parameter that is set in lib/galaxy/managers/configuration.py: ... # TODO: this doesn't seem right 'logo_url' : lambda i, k, **c: self.url_for( i.get( k, '/' ) ), 'logo_src' : lambda i, k, **c: self.url_for( '/static/images/galaxyIcon_noText.png' ), ... However I have no idea how to try and fix it - presumably it needs to incorporate the proxy-prefix setting somehow. (Btw the 'logo_url' does seem to point to the correct location however.) Can someone suggest a fix? Thanks Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy 16.07: datasets don't update to yellow "running" state with novel job runner
Dear Galaxy devs (Apologies: this is a repost of my message from last week, with a correction: the problem actually appears in the current Galaxy release 16.07.) I've recently updated our local Galaxy instances to use a novel in-house job runner (as part of a migration to a new VM platform). In all versions of Galaxy pre-16.07, the runner executes without any problems. However in 16.07 there is a minor bug in the UI, where jobs never update from the grey "queued" state to the yellow "running" state. The jobs themselves run okay in all versions. Does anyone know if there were changes in 16.07 which might have caused this? I couldn't see anything in the change log, or by comparing with the distributed runners. (The our runner code can be seen here: https://github.com/pjbriggs/ansible-palfinder-galaxy/blob/jse-drop-add-prefix-option/roles/galaxy/files/jse_drop_runner.py with the implementation of the 'check_watched_item' method being the likely culprit?) Any suggestions immensely appreciated! Best wishes Peter From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Peter Briggs [peter.bri...@manchester.ac.uk] Sent: Friday, December 02, 2016 2:58 PM To: galaxy-dev@lists.galaxyproject.org Subject: [galaxy-dev] Datasets no longer automatically update to yellow "running" state using novel job runner Hello all I've recently updated our local Galaxy instances to use a novel in-house job runner as part of a migration to a new VM platform. As far as I can determine the runner is able to run jobs without any issues, on all versions of Galaxy that I've tried (v15.10/v16.01/v16.04/release_16.10). However: when moving to release_16.10 I've noticed a minor issue with the UI, where the job state in the history no longer automatically switches from the grey "queued" state to the yellow "running" state - even though the logs indicate that the job status has changed internally. Refreshing the history correctly updates the job to the correct yellow "running" state (so it would seem that internally Galaxy knows that the job status has changed), and transitions between other states seem to happen automatically. Comparing my novel runner code with that for e.g. the drmaa runner , I can't see what might be missing from mine to cause this issue. I had wondered if it could be a problem with out of date Javascript in the UI somehow not detecting the job status change from "queued" to "running", but using the "local" job runner doesn't exhibit this behaviour. (The source code for the runner can be found here: https://github.com/pjbriggs/ansible-palfinder-galaxy/blob/jse-drop-add-prefix-option/roles/galaxy/files/jse_drop_runner.py It submits jobs to the cluster by "dropping" the scripts into a shared directory, which are then collected and executed by a separate process running on the cluster - with all communication done via files.) Can anyone suggest what might be happening to cause this issue? Many thanks Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Datasets no longer automatically update to yellow "running" state using novel job runner
Hello all I've recently updated our local Galaxy instances to use a novel in-house job runner as part of a migration to a new VM platform. As far as I can determine the runner is able to run jobs without any issues, on all versions of Galaxy that I've tried (v15.10/v16.01/v16.04/release_16.10). However: when moving to release_16.10 I've noticed a minor issue with the UI, where the job state in the history no longer automatically switches from the grey "queued" state to the yellow "running" state - even though the logs indicate that the job status has changed internally. Refreshing the history correctly updates the job to the correct yellow "running" state (so it would seem that internally Galaxy knows that the job status has changed), and transitions between other states seem to happen automatically. Comparing my novel runner code with that for e.g. the drmaa runner , I can't see what might be missing from mine to cause this issue. I had wondered if it could be a problem with out of date Javascript in the UI somehow not detecting the job status change from "queued" to "running", but using the "local" job runner doesn't exhibit this behaviour. (The source code for the runner can be found here: https://github.com/pjbriggs/ansible-palfinder-galaxy/blob/jse-drop-add-prefix-option/roles/galaxy/files/jse_drop_runner.py It submits jobs to the cluster by "dropping" the scripts into a shared directory, which are then collected and executed by a separate process running on the cluster - with all communication done via files.) Can anyone suggest what might be happening to cause this issue? Many thanks Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Moving Galaxy Install From One Server To Another
Hi Peter Thanks for your comments - at least it doesn't sound like I'm missing the "perfect" way to migrate the installed tools. In the longer term moving towards bioconda dependency resolution will probably be more stable. Also re breaking tool repos: even repos that aren't updated on the toolshed can break over time, for example if the URL for an executable moves, or if an implicit Python dependency is updated and breaks an install (these are both things that I'm seeing in my tests). Thanks again Best wishes Peter On 23/11/16 11:40, Peter Cock wrote: On Wed, Nov 23, 2016 at 10:49 AM, Peter Briggs <peter.bri...@manchester.ac.uk> wrote: Hello Evan, Hans-Rudolf I'm just in the middle of doing a similar migration for our local production server, and Hans-Rudolf's advice seems sound to me. Definitely moving the core Galaxy server has been relatively straightforward. However: for the installed tools, I'm not sure that changing the paths in the env.sh files is sufficient - in our installation the absolute paths seemed to be baked into a lot of other files under the 'tool_dependencies' directory - including things like compiled files (e.g. static and shared libraries). So for many of the tools I wouldn't feel confident that they would still work after the move. Yes, I've seen that with various third party tool installations (out side of Galaxy), so sadly you cannot in general move the files to a different path :( I don't know if we can do anything about this directly in Galaxy, or even in BioConda? My plan has been to reinstall each of the tools from the toolshed (i.e. uninstall via the admin interface then reinstall the same tool repository revision(s) using the API), but I don't feel able to recommend this approach either as in my testing this has also had problems - ranging from some tool revisions no longer being available, through to more serious issues (such as tool dependencies which used to work but since become broken). I figured I'd just have to knuckle down and work through each problem as I encountered it. One simple reason for this is stale URLs, where Galaxy's cache can help but is another step for tool wrapper authors to do when setting up a new Galaxy dependency: https://github.com/galaxyproject/cargo-port However, even well intentioned Tool Shed updates could also break things :( If anyone else has experiences with these kinds of migrations then I'm also very interested to know what worked (and what didn't)! Sorry - the closest I've been is setting up a new Galaxy server in parallel with our old server, and manually installing "missing" tools via the Tool Shed. Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] how to create users through Galaxy API?
Hello Peter Following on from Nicola's suggestion: a while ago I had a go at writing some command line tools using Bioblend to try and do some of my Galaxy admin tasks via the API, including creating new user accounts so I'd also recommend this route as much much easier than attempting to interact with the API directly. The tools I wrote are here if you'd like to look at some an example of using Bioblend for this, to get you started: https://github.com/pjbriggs/nebulizer/blob/master/nebulizer/users.py As long as the user that the API key belongs to is an admin user (or if you're using a master API key) this works fine. HTH Best wishes Peter On 29/08/16 17:37, Nicola Soranzo wrote: Hi Peter, why not use BioBlend? https://bioblend.readthedocs.io/en/latest/api_docs/galaxy/all.html#module-bioblend.galaxy.users Cheers, Nicola Peter Cock ha scritto Thanks Hans-Rudolf, Yes - you are right. The wiki does link there but it wasn't obvious to me. Reading the UserAPIController link source code suggests as long as I am authenticated as a Galaxy Admin, I could use this to create new user accounts: https://docs.galaxyproject.org/en/master/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create Thank looks very promising... Peter On Mon, Aug 29, 2016 at 4:36 PM, Hans-Rudolf Hotz <h...@fmi.ch> wrote: Hi Peter I guess it is here: https://docs.galaxyproject.org/en/master/api_doc.html Regards, Hans-Rudolf On 08/29/2016 05:27 PM, Peter Cock wrote: Hi Martin, I'd like to look into creating new users via the API (where we can control the email address and username format for integration with our cluster), and have the web-interface forbid users from self-registration. The read-the-docs site you mentioned in this old email (below) doesn't exist anymore, and the API wiki page doesn't seem to link to something similar: https://wiki.galaxyproject.org/Admin/API Where is the official API documentation now? Thanks, Peter On Mon, Dec 30, 2013 at 4:34 AM, Martin Čech <mar...@bx.psu.edu <mailto:mar...@bx.psu.edu>> wrote: Hi, you specify username, password and email in the body (payload) of the POST as Key:Value pairs. Code from the API method: ( https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create <https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create> ) username=payload['username']email=payload['email']password=payload['password'] There are also other conditions that need to be fulfilled (e.g. user creation has to be turned on in the configuration) - you will find these when you look at the source code of the method (because the documentation is not perfect yet, sorry). M. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Building tool dependencies
Hello Matthias From what I've seen there are a number of different recipes used for installing Python packages via the toolshed; the approach you're using seems to be pretty common. In these cases the usual 'shell_command' seems to be: python setup.py install --install-lib $INSTALL_DIR/lib/python --install-scripts $INSTALL_DIR/bin which should avoid installing to /usr/bin. You'll then need to do something like: name="PYTHONPATH">$INSTALL_DIR/lib/python name="PATH">$INSTALL_DIR/bin ... to ensure that the correct environment is set up when the tool is used. Aside from this: elsewhere I've seen other approaches including a "magic" "package" command where the whole download and install is taken care of in a single tag. For example: ... sha256sum="e01853dfe111f3aea005315573400b7216ddbabbf1f28d482a71217d67ae4f81">https://pypi.python.org/packages/source/p/pandas/pandas-0.16.2.tar.gz ... (see https://github.com/galaxyproject/tools-iuc/blob/master/packages/package_python_2_7_pandas_0_16/tool_dependencies.xml for the full recipe that I've pulled this fragment from.) HTH Best wishes Peter On 05/07/16 07:56, Matthias De Smet wrote: Hi I’m trying to put together a tool + dependency package to use umi-tools (https://github.com/CGATOxford/UMI-tools) from Galaxy. I’m at a loss on how I should organise this. The package is available from either github or pip. Which one is preferable to use? How do I make sure the package dependencies are installed? I’ve found the and installing it with python setup.py install. This, however, tries to install the package to /usr/bin/ which isn’t desirable (obviously). Thanks a lot! M ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Deploying custom genome in Galaxy for use with ANNOVAR
Hello David I don't think it will answer all your questions, however: I wrote up my experiences with installing ANNOVAR in Galaxy last year which might be useful as a starting point: http://galacticengineer.blogspot.co.uk/2015/03/installing-annovar-in-galaxy.html As I recall all the databases we installed were standard ones from ANNOVAR. However the above instructions might be useful in at least getting started. Apologies if I've misunderstood your question and this doesn't help, Best wishes Peter On 07/06/16 17:48, David Kovalic wrote: All, From my read of the command line docs for ANNOVAR I believe that the program needs so ancillary files for its operation (e.g. GenePred file, transcripts fasta file, etc...). Does anyone have any experience deploying custom genomes in Galaxy for use with ANNOVAR? Specifically what files need to go where in order to enable the Galaxy ANNOVAR front end? Thanks, David ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unsorted BAM file --> Galaxy crash?
Hello Ray Just to clarify, for us it wasn't dependent on the tool - it was actually the version of samtools installed in the 'galaxy' user's environment i.e. on the 'galaxy' user's PATH. I set it explicitly by creating a file called "local_env.sh" in the 'config' directory of the Galaxy installation with the following content: $ cat local_env.sh # Prepend samtools 1.2 export PATH=$HOME/apps/samtools/1.2/bin:$PATH ## # which is automatically picked up by 'run.sh' when Galaxy is started (I don't know how that plays if you're using a uwsgi/supervisor setup). Best wishes Peter On 24/05/16 11:46, Raymond Wan wrote: Hi Peter, On Tue, May 24, 2016 at 6:29 PM, Peter Briggs <peter.bri...@manchester.ac.uk> wrote: One of my colleagues encountered a similar-sounding error on our local test instance, where an unsorted BAM file seemed to crash the Galaxy handler processes and prevented them from restarting. In our case the default samtools version in the Galaxy environment was 0.1.18; we found that updating this to samtools 1.2 fixed the crash problem. Ah! That does sound familiar. As I just replied in another message, we seem to have at least 5 versions of samtools in the system. I'm not sure if that is normal or not for a Galaxy instance... Let me see if I can figure out (through logs, but most likely by asking the user) what tool he was using and figure out which samtools executable was being run. If it is 0.1.18, I'll look into update it...thank you for sharing your colleague's experience! Ray -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unsorted BAM file --> Galaxy crash?
Hello Ray One of my colleagues encountered a similar-sounding error on our local test instance, where an unsorted BAM file seemed to crash the Galaxy handler processes and prevented them from restarting. In our case the default samtools version in the Galaxy environment was 0.1.18; we found that updating this to samtools 1.2 fixed the crash problem. Hope this helps, Best wishes Peter On 24/05/16 11:07, Raymond Wan wrote: Dear all, We are experiencing a crash of our Galaxy instance (latest git log is early January 2016) and it seems unable to restart. The last entries in paster.log says the following: galaxy.web.framework.base DEBUG 2016-05-24 17:46:42,593 Enabling 'histories' API controller, class: HistoriesController galaxy.web.framework.base DEBUG 2016-05-24 17:46:42,622 Enabling 'history_contents' API controller, class: HistoryContentsController [bam_index_core] the alignment is not sorted (XXX): 21-th chr > 11-th chr [bam_index_build2] fail to index the BAM file. galaxy.web.framework.base DEBUG 2016-05-24 17:46:42,632 Enabling 'history_content_tags' API controller, class: HistoryContentTagsController [bam_index_core] the alignment is not sorted (XXX): 24-th chr > 9-th chr [bam_index_build2] fail to index the BAM file. galaxy.datatypes.metadata DEBUG 2016-05-24 17:46:42,638 setting metadata externally failed for HistoryDatasetAssociation 442: External set_meta() not called galaxy.jobs.runners.slurm WARNING 2016-05-24 17:46:42,642 (313/189770) Job not found, assuming job check exceeded MinJobAge and completing as successful So, the system cannot start and if we visit the site, it says "503 Service Temporarily Unavailable". The log file seems to indicate some problem with the sorting of the BAM file. The problem looks a bit like something from 9 months ago (https://www.biostars.org/p/159336/), but the messages I've seen do not indicate that the problem prevents Galaxy from starting up. I would have thought if there is a problem with the input file, the job would have failed, but not affect Galaxy from starting up again. So, I'm wondering if that is the problem or maybe it's the line after about, "setting metadata externally failed" Has anyone seen this problem before or might have an idea about what to do about it? Thank you in advance! Ray ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Using forms to collect user info at registration
Hello
I'm trying to set up a form within Galaxy to collect some user info at
registration time. The only documentation I've found on this
functionality is:
https://wiki.galaxyproject.org/Admin/Config/User%20Information
which seems to be precisely what I need.
However I can't make this work as described. When I set up a form and
then go to the registration page, instead of the additional form options
I get a single field called "User type" with a drop-down menu widget
with a single option ("User registration form" i.e. the name of the form
that I created).
I've looked to see if there is a config option to turn something on, or
if I need to create a role, but so far nothing I've tried seems to make
a difference.
Does anyone have any suggestions on how I might get this feature to
operate as advertised? (Btw I'm seeing this in both 15.10 and 16.01.)
Thanks for your help,
Best wishes
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Missing dependency issue
Hello Scott I don't have any insight into your specific problem with the readline package, however it might be worth looking for the INSTALLATION.log file from the tool install process. Assuming that this is the iuc version of package_readline_6_2 then you can try looking for it under .../tool_dependencies/readline/6.2/iuc/package_readline_6_2/52d6f646b3b5/ (the final subdirectory corresponds to the version hash so might be different for your installation). If present the INSTALLATION.log file should be in there and hopefully will give you some clues as to what the problem could be. HTH Best wishes Peter On 09/03/16 20:33, Scott Szakonyi wrote: Hello all, I'm having an issue with a missing dependency for readline 6.2. When I try to install or repair, I get the following error message: configure: error: in `/vectorbase/web/Galaxy/galaxy/data/galaxy/tmp/tmp-toolshed-mtd3fhemK/readline-6.2': configure: error: cannot compute suffix of object files: cannot compile See `config.log' for more details. I've tried to locate the config.log file without success, so this is all I have to go on. Has anyone else encountered this sort of issue, and if so, how did you solve it? Thanks! -- Scott B. Szakonyi Research Programmer *Center for Research Computing* 107 Information Technology Center Notre Dame, IN 46556 http://crc.nd.edu <http://crc.nd.edu/> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Setting job_working_directory on per-tool basis? (cuffdiff/cummerbund SQLite db error)
Just to follow up on my own questions, in case it's of interest to others: -- Re specifying different job working directories on a per-tool basis: Having spent a bit of time looking at the "advanced" sample job_conf.xml and also at the code for setting up and dispatching jobs, it looks to me like it is not possible to do this (i.e. specify different job working directories within different job destinations). In fact as far as I can tell (I'm slightly lost within the Galaxy code at this point), it looks like the working directory is created within the job wrapper (via the objectstore interface) before the handler is assigned. I'm wondering if a possible workaround might be to use Pulsar running on the same server, but it looks like a lot of overhead for my current problem (especially as I don't have any experience currently with using Pulsar). -- Re cuffdiff cummeRbund SQLite database error: It looks like changing the job_working_directory to a non-lustre filesystem fixes this problem. Best wishes Peter On 08/03/16 09:37, Peter Briggs wrote: Hello I've encountered an issue with running the cuffdiff tool on our local production Galaxy instance. Specifically when a cummerbund SQLite database is requested as output then the tool fails with an error: Creating database ./cummeRbund.sqlite Error in sqliteSendQuery(con, statement, bind.data) : error in statement: disk I/O error Error in sqliteSendQuery(con, statement, bind.data) : error in statement: disk I/O error Calls: readCufflinks ... .local -> sqliteGetQuery -> sqliteSendQuery -> .Call Execution halted Our Galaxy is configured to use a LustreFS scratch directory for the 'job_working_directory', which I understand is where each job will have its own working directory created, and I'm wondering if the SQLite error above is a manifestation of a general problem with SQLite on LustreFS (see e.g. the answer to this BioStars question https://www.biostars.org/p/115452/). The obvious workaround is to change 'job_working_directory', preferably just for the cuffdiff tool. Does anyone know if it's possible to do this i.e. set 'job_working_directory' on a per-tool basis? (Also, has anyone else seen this specific problem with cuffdiff/cummerbund/SQLite on their local Galaxy instances?) Thanks for your help - any suggestions greatly appreciated, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Setting job_working_directory on per-tool basis? (cuffdiff/cummerbund SQLite db error)
Hello I've encountered an issue with running the cuffdiff tool on our local production Galaxy instance. Specifically when a cummerbund SQLite database is requested as output then the tool fails with an error: Creating database ./cummeRbund.sqlite Error in sqliteSendQuery(con, statement, bind.data) : error in statement: disk I/O error Error in sqliteSendQuery(con, statement, bind.data) : error in statement: disk I/O error Calls: readCufflinks ... .local -> sqliteGetQuery -> sqliteSendQuery -> .Call Execution halted Our Galaxy is configured to use a LustreFS scratch directory for the 'job_working_directory', which I understand is where each job will have its own working directory created, and I'm wondering if the SQLite error above is a manifestation of a general problem with SQLite on LustreFS (see e.g. the answer to this BioStars question https://www.biostars.org/p/115452/). The obvious workaround is to change 'job_working_directory', preferably just for the cuffdiff tool. Does anyone know if it's possible to do this i.e. set 'job_working_directory' on a per-tool basis? (Also, has anyone else seen this specific problem with cuffdiff/cummerbund/SQLite on their local Galaxy instances?) Thanks for your help - any suggestions greatly appreciated, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Broken package_perl_5_18, was Re: JBrowse on dev branch: progress
Hello Peter
Re the failing install of 'package_perl_5_18', I've investigated further
now and have submitted a patch which seems to fix the problem:
https://github.com/galaxyproject/tools-iuc/pull/616
Not sure if this is of any immediate help,
Best wishes
Peter
On 22/02/16 09:43, Peter van Heusden wrote:
Ok, second attempt, perl install died with the same error, so I suspect
you're right about the cause. Eric?
Peter
On 22 February 2016 at 11:41, Peter van Heusden <p...@sanbi.ac.za
<mailto:p...@sanbi.ac.za>> wrote:
Thanks Peter
I managed, on the second attempt, to get past the gnu coreutils
problem. Zipho noticed a weird lockup though: conftest that seemed
to be from the gnu coreutils install wanted to listen on port 4001,
which is the same port our uwsgi server listens on. The install has
now completed but I'll try and find time to go back and verify this.
On the second attempt perl seems to be installing, so I'll see if it
completes or hits the error you're mentioning.
Peter
On 22 February 2016 at 11:34, Peter Briggs
<peter.bri...@manchester.ac.uk
<mailto:peter.bri...@manchester.ac.uk>> wrote:
Hi Peter
I think that package_perl_5_18 in the tool shed is broken, I
came across the same error last week on both our production
instance and a local 'test' instance (both v15.10).
My suspicion is that the following lines are to blame:
...
http://www.cpan.org/authors/id/A/AP/APEIRON/local-lib-1.008009.tar.gz
tar xfvz
local-lib-1.008009.tar.gz
...
Looking at the Galaxy internals I believe that the
'download_file' action automatically unpacks the archive and
cd's into the resulting directory, hence the failure for the
subsequent 'tar xfvz ...' command. But I haven't had the time
yet to verify this or create a patch to see if the removal of
this line addresses the problem.
I don't think I've seen the problem with
'package_gnu_coreutils_8_22' that you describe however so it may
be that my hypothesis is not correct.
HTH
Best wishes
Peter
On 22/02/16 05:10, Peter van Heusden wrote:
Hi there Eric and others
I tried to install the JBrowse tool on a dev branch server
running on an
Ubuntu 14.04 VM.
It got stuck at the package_gnu_coreutils_8_22 dependency
(revision
ac64dfe4b1fb) - this ended up in "Installing tool
dependencies" state.
If I go look at the tool details it highlights errors
in package_perl_5_18 which in turn refers to errors in
gnu_coreutils
(just "Error" state) and then this more informative failure
for perl:
"tar (child): local-lib-1.008009.tar.gz: Cannot open: No
such file or
directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now"
I wonder if this is a sign of a download error?
Digging a bit more I find the error "Shutting down process
id 21094
because it generated no output for the defined timeout
period of 3600.0
seconds." for gnu_coreutils.
I'm trying to do a reinstall of the gnu_coreutils and perl
packages to
see if I can get further.
Peter
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Peter Briggs peter.bri...@manchester.ac.uk
<mailto:peter.bri...@manchester.ac.uk>
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Re: [galaxy-dev] Trimmomatic doesn't recognise pair collection
Hello Peter
Glad that you found the problem, thanks for the update.
Best wishes
Peter
On 04/02/16 12:20, Peter van Heusden wrote:
Hi Peter
We found that it was a problem in the Galaxy code - it was fixed in the
some of the patches that went towards the 16.01 release.
Thanks,
Peter
On 4 February 2016 at 13:47, Peter Briggs <peter.bri...@manchester.ac.uk
<mailto:peter.bri...@manchester.ac.uk>> wrote:
Hello Peter
I'm the author of the Trimmomatic tool on the toolshed (i.e.
https://toolshed.g2.bx.psu.edu/view/pjbriggs/trimmomatic) -
apologies for not responding sooner.
The tool works for me on our production server (v15.10 from github)
when using a pair of fastqs as input, by which I mean that it allows
me to select a fastq-pair dataset as input and produces 2 fastq-pair
datasets. (I'm haven't tried on a list of pairs.)
What do you see when you say that the tool 'is not working'? I'm
wondering if it is a problem with the tool itself, or something to
do with the underlying Galaxy infrastructure when using the head of
the dev branch).
Best wishes
Peter
On 22/01/16 13:53, Peter van Heusden wrote:
Hi there
I've got Trimmomatic installed (from the toolshed) into our
local Galaxy
server (server is latest commit on the dev branch from GitHub). It
refuses to see a dataset pair as a valid input, however.
I've tested this with two types: firstly a pair made out of 2
individual
datasets (both fastqsanger) and then also a list of pairs. I've
tested
this with 'planemo serve' and the tool works, but for some
reason it is
not working on our server. Any ideas?
Thanks,
Peter
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Peter Briggs peter.bri...@manchester.ac.uk
<mailto:peter.bri...@manchester.ac.uk>
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] planemo reports test failure when test succeeds
Hello again Peter I also see this when using planemo to run the tool tests both on trimmomatic and also on another of our tools i.e. one or more of the tests will be reported as 'failed' on the command line when the HTML report indicates that all tests executed successfully. Now I think about it, planemo itself must exit with status 0 in these cases as my tests on Travis-CI also report success, for example see: https://travis-ci.org/fls-bioinformatics-core/galaxy-tools/jobs/104643315 Scroll to the bottom of the log and the last test is reported as 'failed', however the Travis test overall passes. I hadn't really thought about it before since the planemo exit status seems to be correct (i.e. in cases where there are genuine tool test failures, it will return a non-zero status at the end which causes Travis to correctly report a failure). (Btw this referenced above used planemo 0.22.2 but I've seen it in older versions too.) I don't think I've ever seen planemo report a 'passed' test as 'failed', but generally I check the HTML to verify whether the failures are real or not. Best wishes Peter On 22/01/16 11:40, Peter van Heusden wrote: Hi there When testing the latest trimmomatic tool (from https://github.com/fls-bioinformatics-core/galaxy-tools/tree/master/trimmomatic, planemo test reports an error on the last test, but according to the test report the test has succeeded (see attached HTML and JSON). For the test setup you need to set some environment variables: TRIMMOMATIC_DIR - the location of the trimmomatic jar TRIMMOMATIC_JAR - the name of the jar file TRIMMOMATIC_ADAPTERS_DIR - location of adapters This is with planemo 0.22.2 and a fresh $HOME/.planemo. Any ideas? Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RedHat versus CentOs
Hello Hans-Rudolf We're running our production Galaxy on Scientific Linux 6.5 here. HTH Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Hans-Rudolf Hotz [h...@fmi.ch] Sent: Wednesday, January 20, 2016 1:20 PM To: galaxy-...@lists.bx.psu.edu Subject: [galaxy-dev] RedHat versus CentOs Dear All Our production Galaxy server has been running smoothly on a single multicore box with Red Hat 6.7. As this box gets older and older, it is in now time to replace it. At the same time, we are also considering switching the operating system to CentOS 7. Has anyone switched a production installation from Red Hat to CentOS ? 'Galaxy Main' is running on CentOS, isn't-it? I am also interested in plain numbers: how many of you are using Red Hat, how many CentOS and which version for your production server. I will probably need such numbers to convince our IT department. There is no need to reply to the mailing list, just reply to me (h...@fmi.ch) and if I get enough I will then collate and anonymize all the data and post back. Maybe we can even add such information to the wiki (https://wiki.galaxyproject.org/Community/Deployments)? Thank you very much for your help Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CEAS error
Hello Philipp I'm assuming this is the 'ceas' tool from the toolshed which is owned by 'pjbriggs' (that is me)? If it is then from what you describe I'm not sure whether this is a problem with the specific tool or a more general issue with Galaxy's tool installation. My own experiences of trying to debug tool dependency installation issues have been quite haphazard, and although I've seen this particular error (i.e the "database had no record") a few times with various tools, I've never been able to understand what has caused it. First it may be worth trying the "Repair repository" option from the "Repository Actions" menu. If that doesn't help then I've also had some success in the past using the "Manage tool dependencies" option (which allows individual dependencies to be uninstalled and then re-installed). This might work if just one dependency is giving an error. Beyond that I've found it necessary to start drilling into the Galaxy log files. Aside from the log of the actual Galaxy process (verbose but sometimes useful, I don't know how easy this is to find on your Docker instance - I don't have any experience of working with them), often the tool dependencies also create their own INSTALLATION.log files which are worth looking for/at. The following recipe should allow you to access these from within Galaxy (this is for v15.10): - Go to "Admin" -> "Manage installed tools" - Click on the name of the tool of interest (e.g. 'ceas') - Click on the name of an errored dependency under the "Missing tool dependencies" section - In the list of dependencies, click on the name of the errored dependency again This should take you to a page which includes a "folder browser" underneath the heading "Contents", where you can browse the contents of the installed tool dependency folder. Hopefully this will include a file called INSTALLATION.log, and if you click on it then the contents should be displayed and you can look to see if there are any error messages to indicate what went wrong. Sometimes there is no INSTALLATION.log file however, in which case the main log file of the Galaxy process is probably the next place to look (if possible). Sorry I don't have any more definitive answers - please let me know if any of the above helps. Best wishes Peter On 17/01/16 18:36, Rathert, Philipp, Dr. wrote: Dear All, We are running a local docker-galaxy-stable instance on a Mac OS X system. and observe a strange error: CEAS Description: Revision:f411ce97a351 is not installing correctly. After installing mysql-python in the container directly basically every package is installed: Installed tool dependencies/ - click the name to browse the dependency installation directory/ NameVersion TypeInstallation status R <http://129.69.102.42/admin_toolshed/browse_tool_dependency?id=52d6bdfafedbb5e5> 3.1.2 package Installed ucsc_fetchChromSizes <http://129.69.102.42/admin_toolshed/browse_tool_dependency?id=ae9879a80bd0a7bc> 1.0 package Installed python_mysqldb <http://129.69.102.42/admin_toolshed/browse_tool_dependency?id=03552bdee8c8190f> 1.2.5 package Installed bx_python <http://129.69.102.42/admin_toolshed/browse_tool_dependency?id=f6509e58756240f2> 0.7.1 package Installed Missing tool dependencies/ - click the name to install the missing dependency/ NameVersion TypeInstallation status cistrome_ceas <http://129.69.102.42/admin_toolshed/manage_repository_tool_dependencies?tool_dependency_ids=71b3098c1ce064d4> 1.0.2.d8c0751 package Error cistrome_ceas <http://129.69.102.42/admin_toolshed/manage_repository_tool_dependencies?repository_id=be0a27b9edd0db03=browse_dependency_ids=71b3098c1ce064d4> 1.0.2.d8c0751 package Error The installation directory for this tool dependency had contents but the database had no record. The installation log may show this tool dependency to be correctly installed, but due to the missing database record it is now being set to Error. A reinstall results in the same error on different dependencies and it looks like a random event. Does anybody have an idea what could solve the issue? I hope I provided enough information. In your answer please keep in mind I am a newcomer to all this and a wet lab scientist. Thank you for your help. Cheers, philipp ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 27514
Re: [galaxy-dev] python_mysqldb caused zip_safe flag not set
Hello Mic Apologies, I missed your post the first time around - I didn't pick up that it was related to my 'ceas' tool. However the error you reported sounds superficially similar to that reported by Philipp Rather. Did you manage to resolve the problem yourself? If not (and if you're still interested in using the tool) then I'd first suggest trying to install the version from the main toolshed and seeing if you get the same error. (I don't think that the 'zip_safe flag not set' message in the INSTALLATION.log file is the problem as I also see this message in the STDERR section for successful installs.) Best wishes Peter On 10/11/15 01:47, Mic wrote: Hello, I tried to install CEAS from testtoolshed, but I have recieved the below error from python_mysqldb (tool_dependency/python_mysqldb/1.2.5/pjbriggs/ceas/82df5af2995f) The installation directory for this tool dependency had contents but the database had no record. The installation log may show this tool dependency to be correctly installed, but due to the missing database record it is now being set to Error. I installed MariaDB 10.1 on Ubuntu 14.04 in the following way: apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 <http://keyserver.ubuntu.com:80> 0xcbcb082a1bb943db add-apt-repository 'deb [arch=amd64,i386] http://mirror.aarnet.edu.au/pub/MariaDB/repo/10.1/ubuntu trusty main' apt-get update apt-get install mariadb-server libmariadbclient-dev The INSTALLATION.log shows the following information: ... byte-compiling build/bdist.linux-x86_64/egg/MySQLdb/cursors.py to cursors.pyc creating stub loader for _mysql.so byte-compiling build/bdist.linux-x86_64/egg/_mysql.py to _mysql.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying MySQL_python.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying MySQL_python.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying MySQL_python.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt creating dist removing 'build/bdist.linux-x86_64/egg' (and everything under it) Creating /home/lorencm/galaxy/tool_dependency/python_mysqldb/1.2.5/pjbriggs/ceas/82df5af2995f/lib/python/site.py Processing MySQL_python-1.2.5-py2.7-linux-x86_64.egg Copying MySQL_python-1.2.5-py2.7-linux-x86_64.egg to /home/lorencm/galaxy/tool_dependency/python_mysqldb/1.2.5/pjbriggs/ceas/82df5af2995f/lib/python Adding MySQL-python 1.2.5 to easy-install.pth file Installed /home/lorencm/galaxy/tool_dependency/python_mysqldb/1.2.5/pjbriggs/ceas/82df5af2995f/lib/python/MySQL_python-1.2.5-py2.7-linux-x86_64.egg Processing dependencies for MySQL-python==1.2.5 Finished processing dependencies for MySQL-python==1.2.5 # # export PYTHONPATH=$PYTHONPATH:/home/lorencm/galaxy/tool_dependency/python_mysqldb/1.2.5/pjbriggs/ceas/82df5af2995f/lib/python && python setup.py install --install-lib /home/lorencm/galaxy/tool_dependency/python_mysqldb/1.2.5/pjbriggs/ceas/82df5af2995f/lib/python --install-scripts /home/lorencm/galaxy/tool_dependency/python_mysqldb/1.2.5/pjbriggs/ceas/82df5af2995f/bin STDERR zip_safe flag not set; analyzing archive contents... What did I miss? Thank you in advance. Mic ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Bug in galaxy-dist/latest_15.05 for Share history with user function
Hello I think a bug was introduced into galaxy-dist between the 15.03 and 15.05 releases, for the Share history with user function. The function works for 15.03 but fails in 15.05. To reproduce: 1. Click on the cog icon for a history and select Share or publish 2. Under Share History with Individual Users click Share with a user 3. Enter the email of another user on the system and click submit In 15.03 this returns to the sharing screen and indicates that the history has been shared with the specified user; in 15.05 this produces a stack trace (or internal server error in a production setting). The following patch appears to fix the problem: diff -r c3cef260df88 lib/galaxy/webapps/galaxy/controllers/history.py --- a/lib/galaxy/webapps/galaxy/controllers/history.py Fri May 15 15:20:17 2015 -0400 +++ b/lib/galaxy/webapps/galaxy/controllers/history.py Wed Jun 17 12:51:41 2015 +0100 @@ -899,7 +899,7 @@ for history_id in ids: history_id = self.decode_id( history_id ) history = self.history_manager.get_owned( history_id, trans.user, current_history=trans.history ) -histories.append( ) +histories.append( history ) return histories def _get_users( self, trans, user, emails_or_ids ): also available via this gist: https://gist.github.com/pjbriggs/65c29a87e094d4d257f6 Apologies if this has already been reported/fixed elsewhere, or if there's a better way to submit this fix (if so then please let me know) Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bug in galaxy-dist/latest_15.05 for Share history with user function
Hello Dannon Thanks for the quick response - glad to hear it's already been detected and fixed. Just to confirm though - am I right in thinking that the fix hasn't propagated through to bitbucket/galaxy-dist yet? Our production instance is still based off this and the change didn't seem to come up when I did hg update to the tip. Should we be considering migrating our deployment to github at some point? Thanks again, best wishes Peter On 17/06/15 14:12, Dannon Baker wrote: Hey Peter, this was indeed reported and fixed in https://github.com/galaxyproject/galaxy/commit/7294bd0af33bc974684c8ed8d21e97017660ca22, which should already be available in the release_15.05 branch of the repository if you update. Sorry for the trouble, and even though it's resolved now, thanks for taking the time to debug this and write such a detailed error report including a patch! -Dannon On Wed, Jun 17, 2015 at 9:03 AM Peter Briggs peter.bri...@manchester.ac.uk mailto:peter.bri...@manchester.ac.uk wrote: Hello I think a bug was introduced into galaxy-dist between the 15.03 and 15.05 releases, for the Share history with user function. The function works for 15.03 but fails in 15.05. To reproduce: 1. Click on the cog icon for a history and select Share or publish 2. Under Share History with Individual Users click Share with a user 3. Enter the email of another user on the system and click submit In 15.03 this returns to the sharing screen and indicates that the history has been shared with the specified user; in 15.05 this produces a stack trace (or internal server error in a production setting). The following patch appears to fix the problem: diff -r c3cef260df88 lib/galaxy/webapps/galaxy/controllers/history.py --- a/lib/galaxy/webapps/galaxy/controllers/history.py Fri May 15 15:20:17 2015 -0400 +++ b/lib/galaxy/webapps/galaxy/controllers/history.py Wed Jun 17 12:51:41 2015 +0100 @@ -899,7 +899,7 @@ for history_id in ids: history_id = self.decode_id( history_id ) history = self.history_manager.get_owned( history_id, trans.user, current_history=trans.history ) -histories.append( ) +histories.append( history ) return histories def _get_users( self, trans, user, emails_or_ids ): also available via this gist: https://gist.github.com/pjbriggs/65c29a87e094d4d257f6 Apologies if this has already been reported/fixed elsewhere, or if there's a better way to submit this fix (if so then please let me know) Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk mailto:peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bug in galaxy-dist/latest_15.05 for Share history with user function
Hello Bjoern Thanks for that - it doesn't sound too difficult so I'll take a look at doing it when time allows. Much appreciated, best wishes Peter On 17/06/15 15:06, Bjoern Gruening wrote: Hi Peter, it seems that bugfixes are not backported to bitbucket. We had the same trouble here and I migrated to git real quick. It is pretty easy. Check out 15.05 in bitbucket and clone one github repo with the same branch. Now you can swap the .hg and .git folders. Make a backup of everything before :) And do not forget to copy the .gitignore over :) Hope this hleps! Bjoern Hello Dannon Thanks for the quick response - glad to hear it's already been detected and fixed. Just to confirm though - am I right in thinking that the fix hasn't propagated through to bitbucket/galaxy-dist yet? Our production instance is still based off this and the change didn't seem to come up when I did hg update to the tip. Should we be considering migrating our deployment to github at some point? Thanks again, best wishes Peter On 17/06/15 14:12, Dannon Baker wrote: Hey Peter, this was indeed reported and fixed in https://github.com/galaxyproject/galaxy/commit/7294bd0af33bc974684c8ed8d21e97017660ca22, which should already be available in the release_15.05 branch of the repository if you update. Sorry for the trouble, and even though it's resolved now, thanks for taking the time to debug this and write such a detailed error report including a patch! -Dannon On Wed, Jun 17, 2015 at 9:03 AM Peter Briggs peter.bri...@manchester.ac.uk mailto:peter.bri...@manchester.ac.uk wrote: Hello I think a bug was introduced into galaxy-dist between the 15.03 and 15.05 releases, for the Share history with user function. The function works for 15.03 but fails in 15.05. To reproduce: 1. Click on the cog icon for a history and select Share or publish 2. Under Share History with Individual Users click Share with a user 3. Enter the email of another user on the system and click submit In 15.03 this returns to the sharing screen and indicates that the history has been shared with the specified user; in 15.05 this produces a stack trace (or internal server error in a production setting). The following patch appears to fix the problem: diff -r c3cef260df88 lib/galaxy/webapps/galaxy/controllers/history.py --- a/lib/galaxy/webapps/galaxy/controllers/history.py Fri May 15 15:20:17 2015 -0400 +++ b/lib/galaxy/webapps/galaxy/controllers/history.py Wed Jun 17 12:51:41 2015 +0100 @@ -899,7 +899,7 @@ for history_id in ids: history_id = self.decode_id( history_id ) history = self.history_manager.get_owned( history_id, trans.user, current_history=trans.history ) - histories.append( ) + histories.append( history ) return histories def _get_users( self, trans, user, emails_or_ids ): also available via this gist: https://gist.github.com/pjbriggs/65c29a87e094d4d257f6 Apologies if this has already been reported/fixed elsewhere, or if there's a better way to submit this fix (if so then please let me know) Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk mailto:peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Can't set value of conditional boolean parameter in tool tests
Hello John Apologies for the delayed response, but many thanks for looking into this so thoroughly. I've now tried running the test cases using planemo with the dev branch from github and all the cases pass that I expected to. (Using bdg_options.bdg2 rather than bdg_options|bdg2 to reference the parameter in the tests fails consistently, but I understand that this is normal behaviour so it's not a problem.) So it looks to me like you've fixed this. I'm trying it now on my real tool tests where there are a few more moving parts but I don't see why it won't work there. I'll let you know if I do encounter any other strange cases. Thanks again for taking the time to investigate and address these issues, it's greatly appreciated. Best wishes Peter On 20/05/15 20:55, John Chilton wrote: Peter - thanks for the detailed bug report! I believe this should be fixed with the following PR (https://github.com/galaxyproject/galaxy/pull/273) that has been merged into release_15.05 on github as well as the dev branch. I tried to fixed a few different bugs related to boolean and select handling with in the confines of tests - so let me know if it didn't fix this exact use case and I will look at it some more. There is now this test case that verifies the correctness of boolean test parameters inside of conditionals (https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/boolean_conditional.xml) - let me know if you see anything wrong with it. The test can be run with by executing: ./run_tests.sh -framework -id boolean_conditional from Galaxy's root. Thanks again, -John On Tue, May 19, 2015 at 10:01 AM, Peter Briggs peter.bri...@manchester.ac.uk wrote: Hello I've encountered a problem with my tool tests when using a boolean as the control parameter within a conditional: it doesn't appear to be possible to change the value from the default true or false values within the tests. For example if I have a boolean 'bdg2' which controls a conditional block and which defaults to its falsevalue: ... conditional name=bdg_options param name=bdg2 type=boolean checked=False truevalue=-B2 falsevalue= label=Save pileups in bedGraph? / when value=-B2 ... and then try to set the value within a test using: param name=bdg_options|bdg2 value=true / then the value remains unchanged from the default when the tests are run. (Similarly if I set checked=True then the value in the tests is always True regardless of the value I try to set.) I've tried various ways of referencing the parameter (e.g. 'bdg_options|bdg2', 'bdg2' and 'bdg_options') but none of these work for me. Am I doing something obviously wrong? (nb this is a different problem from my previous post about setting boolean values in tests - that works fine for booleans which are not conditional control parameters). I've made a small example tool to investigate the issue, the files are here: https://gist.github.com/pjbriggs/c27e4366dfcdd6d96cb2 and using planemo 0.11.1 to run the tests. (On a side note, it also seems that within the command section, the values returned for this type of boolean only ever appear to be 'True' or 'False' - but it seems okay to match against the truevalue and falsevalues in the when tag.) Thanks for any help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Can't set value of conditional boolean parameter in tool tests
Hello Christian Thanks for taking the time to look at this issue, and for your suggestions - apologies for the delayed response. From the context of the current Galaxy code (and the default used by planemo's --install_galaxy option) I think you're right: i.e. omitting the truevalue/falsevalue attributes; using value=true/false not True/False in the tool tests; and using the much simpler syntax for testing truthiness in the command section, seems to work for me. It looks that John's fixes will address this in future Galaxy releases so that the more general cases will also work. One point: I understand that the correct way to reference nested parameter names within the tool tests is to use a pipe (i.e. |) rather than a dot (.) to separate the name components - that is, param name=bdg_options|bdg2 ... will work but param name=bdg_options.bdg2 ... probably won't. I think that you always need the fully qualified name (using dot notation) to get the values in the command section. Thanks again for looking into this, Best wishes Peter On 21/05/15 10:56, Christian Brenninkmeijer wrote: HI Peter, I forgot to add. In test it is. param name=bdg2 value=true / Do not use the full name as the value is simply ignored This time True/false it is not case sensitive, But in Command it is $bdg_options.bdg2 Do not use the short name as this causes an error Christian University of Manchester From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Christian Brenninkmeijer [christian.brenninkmei...@manchester.ac.uk] Sent: Thursday, May 21, 2015 10:34 AM To: Peter Briggs; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Can't set value of conditional boolean parameter in tool tests Hi Peter, 1. I would not recommend using truevalue= in boolean parm values inside of conditionals are the behaviour is buggy! (I will post a test example in another post) 2, In the when statement the boolean values must be LOWER case as in when value=true 3. In the command the Boolean values are CAPITALIZED as in #if str($bdg_options.bdg2) == 'True' 3a. or better yet do not cast to string but just do #if $bdg_options.bdg2 Christian University of Manchester From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Peter Briggs [peter.bri...@manchester.ac.uk] Sent: Tuesday, May 19, 2015 3:01 PM To: galaxy-dev@lists.galaxyproject.org Subject: [galaxy-dev] Can't set value of conditional boolean parameter in tool tests Hello I've encountered a problem with my tool tests when using a boolean as the control parameter within a conditional: it doesn't appear to be possible to change the value from the default true or false values within the tests. For example if I have a boolean 'bdg2' which controls a conditional block and which defaults to its falsevalue: ... conditional name=bdg_options param name=bdg2 type=boolean checked=False truevalue=-B2 falsevalue= label=Save pileups in bedGraph? / when value=-B2 ... and then try to set the value within a test using: param name=bdg_options|bdg2 value=true / then the value remains unchanged from the default when the tests are run. (Similarly if I set checked=True then the value in the tests is always True regardless of the value I try to set.) I've tried various ways of referencing the parameter (e.g. 'bdg_options|bdg2', 'bdg2' and 'bdg_options') but none of these work for me. Am I doing something obviously wrong? (nb this is a different problem from my previous post about setting boolean values in tests - that works fine for booleans which are not conditional control parameters). I've made a small example tool to investigate the issue, the files are here: https://gist.github.com/pjbriggs/c27e4366dfcdd6d96cb2 and using planemo 0.11.1 to run the tests. (On a side note, it also seems that within the command section, the values returned for this type of boolean only ever appear to be 'True' or 'False' - but it seems okay to match against the truevalue and falsevalues in the when tag.) Thanks for any help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https
[galaxy-dev] Can't set value of conditional boolean parameter in tool tests
Hello I've encountered a problem with my tool tests when using a boolean as the control parameter within a conditional: it doesn't appear to be possible to change the value from the default true or false values within the tests. For example if I have a boolean 'bdg2' which controls a conditional block and which defaults to its falsevalue: ... conditional name=bdg_options param name=bdg2 type=boolean checked=False truevalue=-B2 falsevalue= label=Save pileups in bedGraph? / when value=-B2 ... and then try to set the value within a test using: param name=bdg_options|bdg2 value=true / then the value remains unchanged from the default when the tests are run. (Similarly if I set checked=True then the value in the tests is always True regardless of the value I try to set.) I've tried various ways of referencing the parameter (e.g. 'bdg_options|bdg2', 'bdg2' and 'bdg_options') but none of these work for me. Am I doing something obviously wrong? (nb this is a different problem from my previous post about setting boolean values in tests - that works fine for booleans which are not conditional control parameters). I've made a small example tool to investigate the issue, the files are here: https://gist.github.com/pjbriggs/c27e4366dfcdd6d96cb2 and using planemo 0.11.1 to run the tests. (On a side note, it also seems that within the command section, the values returned for this type of boolean only ever appear to be 'True' or 'False' - but it seems okay to match against the truevalue and falsevalues in the when tag.) Thanks for any help, Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] 'Boolean' parameter type misbehaves in tool tests if truevalue starts with hyphen?
Hello Peter C. Thanks for the prompt response - yes, using 'true' and 'false' in the tests seems to work for my simple example (as do 'True' and 'False'). However I suspect that there is something else going on in the real tool that I'm trying to test, as I still can't get this to work - so I'm going to have to keep investigating. At least now I know that I'm setting the booleans properly in the tests (and will add that to my list of gotchas). Thanks again for your help, best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 From: Peter Cock [p.j.a.c...@googlemail.com] Sent: Friday, May 08, 2015 3:16 PM To: Peter Briggs Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] 'Boolean' parameter type misbehaves in tool tests if truevalue starts with hyphen? On Fri, May 8, 2015 at 3:05 PM, Peter Briggs peter.bri...@manchester.ac.uk wrote: Hello I'm currently writing tests for a tool and think I've found a situation where the behaviour of the boolean parameter type does not seem correct within the test environment. The tool defines the following boolean parameter: param name=spmr type=boolean truevalue=--SPMR falsevalue= checked=False ... / and accesses its value in the command tag using: -B $spmr i.e. if the boolean is true then this should output -B --SPMR, if not then it should output -B . This appears to work as expected when the tool is installed and used interactively. But when running the tests the 'falsevalue' is always returned, even if the value is set to the 'truevalue' in the tests i.e.: param name=spmr value=--SPMR / and param name=spmr value= / both return -B . I think you may need to set the value to true/false here, e.g. https://github.com/peterjc/pico_galaxy/blob/master/tools/samtools_bam2fq/samtools_bam2fq.xml (It would be clearer if the test framework insisted on this, but it may be allowing the truevalue or falsevalue to be used - at least when they don't start with a minus sign?) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] 'Boolean' parameter type misbehaves in tool tests if truevalue starts with hyphen?
Hello I'm currently writing tests for a tool and think I've found a situation where the behaviour of the boolean parameter type does not seem correct within the test environment. The tool defines the following boolean parameter: param name=spmr type=boolean truevalue=--SPMR falsevalue= checked=False ... / and accesses its value in the command tag using: -B $spmr i.e. if the boolean is true then this should output -B --SPMR, if not then it should output -B . This appears to work as expected when the tool is installed and used interactively. But when running the tests the 'falsevalue' is always returned, even if the value is set to the 'truevalue' in the tests i.e.: param name=spmr value=--SPMR / and param name=spmr value= / both return -B . More generally: I've written a small tool to investigate the issue, and it appears that the errant behaviour only manifests when the 'truevalue' starts with a hyphen (e.g. using -Yes is broken, but Yes is okay). The demo tool is available via https://gist.github.com/pjbriggs/5bfbcf22f72e0266e61 and I've been using planemo v0.7.0 to run the tests (planemo t --install_galaxy boolean_test_issue.xml). Have I done something wrong here or is this a genuine bug? Thanks for your help, best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tests not being run on toolsheds?
Hello Dave Thanks for addressing the specific issue of the tests failing to run. Re my initial problem vis-a-vis a tool with tests that worked locally but failed on the toolsheds, this appears to be a result of the two environments using differing versions of perl. This caused the ordering of data in the output files to differ from the reference output in the tests on the toolshed. As the tool wraps a third-party program, I've addressed this by explicitly specifying the version of perl that it uses (at least it seems to have worked for my latest version on the test toolshed - I will update the main toolshed next). The repositories in question are: https://toolshed.g2.bx.psu.edu/view/pjbriggs/pal_finder https://testtoolshed.g2.bx.psu.edu/view/pjbriggs/pal_finder I see a separate problem with a different tool which I'll investigate next and report if I can't understand the toolshed test failure. Thanks again for your help, Best wishes Peter On 18/03/15 13:46, Dave Bouvier wrote: Gentlemen, The issue with the nightly testing was due to a stalled test run blocking subsequent tests. I've cleared out that blockage and a manual test run appears to have completed successfully, as should future automated test runs. As always, feel free to let us know if you encounter any additional inexplicable behavior. --Dave B. On 03/18/2015 07:00 AM, Peter Cock wrote: On Wed, Mar 18, 2015 at 10:13 AM, Peter Briggs peter.bri...@manchester.ac.uk wrote: Hello Peter C. Thanks for your comments and advice Ironically after I sent that email the tests for the next tool I looked at in the toolshed had been run in the past couple of days. That's good. I can also confirm that the Test Tool Shed example I gave was tested overnight, although it looks like a novel failure: https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs I'm interested because I've got a couple of tools that seem to work okay when I run the tests locally, but are reported as failing on the toolshed, and I wanted to see if I'd addressed the issue for one of them by updating the tool dependencies. I suggest asking on this mailing list, posting the shareable Tool Shed URL and a copy of the error message. Hopefully it will be a simple issue (forgetting a test file is my most common mistake), or it may be that another tool author has seen the same error (which can happen due to problems in Galaxy itself). (As an aside I'd also assumed that passing tests is one of the conditions for the tool to be marked as verified, but maybe that's not the case?) I think there is still a human involved, but passing tests is expected for a verified tool. However as you say, more testing locally seems to be a good way to go - thanks for your suggestions. I looked at planemo briefly a while ago and it looked good. The other issue I've had with testing is actually testing tool installation (i.e. tool_dependencies.xml) - I recall that planemo didn't deal with that side of things, so I had to set up the environment for the tests manually. There is some work on tool_dependencies.xml ongoing within planemo, John Chilton would be the person to ask (CC'd directly). Thanks again for your advice and the links, I will investigate trying to set up a local solution (including some CI testing!). This is something where using planemo ought to be very useful - for now I still do my local testing with a test Galaxy instance via: $ ./run_tests.sh -id my_tool_id Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tests not being run on toolsheds?
Thanks Peter. I've reported the details of the specific tool that I was having problems with in my message to Dave B. planemo already looks really good, if it could handle the tool dependencies as well then that would be the icing on the cake. Thanks for your help, best wishes Peter On 18/03/15 11:00, Peter Cock wrote: On Wed, Mar 18, 2015 at 10:13 AM, Peter Briggs peter.bri...@manchester.ac.uk wrote: Hello Peter C. Thanks for your comments and advice Ironically after I sent that email the tests for the next tool I looked at in the toolshed had been run in the past couple of days. That's good. I can also confirm that the Test Tool Shed example I gave was tested overnight, although it looks like a novel failure: https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs I'm interested because I've got a couple of tools that seem to work okay when I run the tests locally, but are reported as failing on the toolshed, and I wanted to see if I'd addressed the issue for one of them by updating the tool dependencies. I suggest asking on this mailing list, posting the shareable Tool Shed URL and a copy of the error message. Hopefully it will be a simple issue (forgetting a test file is my most common mistake), or it may be that another tool author has seen the same error (which can happen due to problems in Galaxy itself). (As an aside I'd also assumed that passing tests is one of the conditions for the tool to be marked as verified, but maybe that's not the case?) I think there is still a human involved, but passing tests is expected for a verified tool. However as you say, more testing locally seems to be a good way to go - thanks for your suggestions. I looked at planemo briefly a while ago and it looked good. The other issue I've had with testing is actually testing tool installation (i.e. tool_dependencies.xml) - I recall that planemo didn't deal with that side of things, so I had to set up the environment for the tests manually. There is some work on tool_dependencies.xml ongoing within planemo, John Chilton would be the person to ask (CC'd directly). Thanks again for your advice and the links, I will investigate trying to set up a local solution (including some CI testing!). This is something where using planemo ought to be very useful - for now I still do my local testing with a test Galaxy instance via: $ ./run_tests.sh -id my_tool_id Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Tests not being run on toolsheds?
Hello Apologies as this seems to be quite a frequent question (and I may have missed something) however: are the tool tests being executed on the main or test toolsheds at the moment? The most recent tests results that I can find for any of the tools I've uploaded is the 29th January. I've also looked at a couple of tools from other users and the situation appears to be the same (at least in those cases). Thanks for your help! Best wishes Peter -- Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
