Dear Galaxy Community,
is there any possibility to expose the command line call, generated within
galaxy?
We searched the net and found this discussions:
https://github.com/galaxyproject/galaxy/issues/3252
https://github.com/galaxyproject/galaxy/issues/2954
But both are on the exposure to non-admin users. Within our instance we see the
command line call nor for admin, neither for normal users!
The problem, behind the question is the error message: "An invalid option was
selected for database, 'myCoolDatabaseName', please verify." In Blast
(toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastn_wrapper/0.2.01)
Local (directly from command line, without Galaxy in between) the blast works
fine.
We are using the 18.05 Docker Galaxy Version.
Thanks in advance!
Mit freundlichen Grüßen
Matthias Enders
-----------------------------------------------------------------------
GERMAN SEED ALLIANCE GmbH
c/o Norddeutsche Pflanzenzucht
Hans-Georg Lembke KG
Hohenlieth, 24363 Holtsee
Tel.: +49 (0)4351/ 736-189
Fax: + 49 (0)4351/ 736-271
Mobil: +49 (0)151/ 14247360
Email: m.end...@german-seed-alliance.de
Firmensitz Köln
Amtsgericht Köln, HRB 73844
-----Ursprüngliche Nachricht-----
Von: galaxy-dev <galaxy-dev-boun...@lists.galaxyproject.org> Im Auftrag von
galaxy-dev-requ...@lists.galaxyproject.org
Gesendet: Montag, 9. Juli 2018 18:00
An: galaxy-dev@lists.galaxyproject.org
Betreff: galaxy-dev Digest, Vol 145, Issue 3
Send galaxy-dev mailing list submissions to
galaxy-dev@lists.galaxyproject.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.galaxyproject.org/listinfo/galaxy-dev
or, via email, send a message with subject or body 'help' to
galaxy-dev-requ...@lists.galaxyproject.org
You can reach the person managing the list at
galaxy-dev-ow...@lists.galaxyproject.org
When replying, please edit your Subject line so it is more specific than "Re:
Contents of galaxy-dev digest..."
HEY! This is important! If you reply to a thread in a digest, please 1. Change the
subject of your response from "Galaxy-dev Digest Vol ..." to the original
subject for the thread.
2. Strip out everything else in the digest that is not part of the thread you
are responding to.
Why?
1. This will keep the subject meaningful. People will have some idea from the
subject line if they should read it or not.
2. Not doing this greatly increases the number of emails that match search
queries, but that aren't actually informative.
Today's Topics:
1. From Paste to uWSGI (Jochen Bick)
2. Re: From Paste to uWSGI (Peter Briggs)
3. Re: From Paste to uWSGI (Martin Čech)
----------------------------------------------------------------------
Message: 1
Date: Mon, 9 Jul 2018 13:49:23 +0200
From: Jochen Bick <jochen.b...@usys.ethz.ch>
To: "galaxy-dev@lists.galaxyproject.org"
<galaxy-dev@lists.galaxyproject.org>
Subject: [galaxy-dev] From Paste to uWSGI
Message-ID: <1a0f33ad-938c-64d5-8db8-28914a5ca...@usys.ethz.ch>
Content-Type: text/plain; charset="utf-8"; format=flowed
Hi all,
we would like to update to 18.05 and thereby also update our Paste Server.
https://docs.galaxyproject.org/en/latest/admin/scaling.html
Is there any tutorial on how to update from Paste to uWSGI?
Cheers Jochen
--
ETH Zurich
*Jochen Bick*
Animal Physiology
Institute of Agricultural Sciences
Postal address: Universitätstrasse 2 / LFW B 58.1
8092 Zurich, Switzerland
Office: Eschikon 27
8315 Lindau, Switzerland
Phone +41 52 354 92 06
jochen.b...@usys.ethz.ch <mailto:jochen.b...@usys.ethz.ch> www.ap.ethz.ch
------------------------------
Message: 2
Date: Mon, 9 Jul 2018 15:31:16 +0000
From: Peter Briggs <peter.bri...@manchester.ac.uk>
To: Jochen Bick <jochen.b...@usys.ethz.ch>,
"galaxy-dev@lists.galaxyproject.org"
<galaxy-dev@lists.galaxyproject.org>
Subject: Re: [galaxy-dev] From Paste to uWSGI
Message-ID:
<e20ab71cd352854b9e24b5830826bc43c8586...@mbxp15.ds.man.ac.uk>
Content-Type: text/plain; charset="iso-8859-1"
Hello Jochen
I'm not aware of a tutorial but here are some observations from my (very)
recent experiences upgrading a local paste-based 17.09 instance to a
uwsgi-based 18.05 configuration. It's not comprehensive because the
configuration will depend on decisions you need to make based on your local
setup, but it might be helpful as a starting point.
Before beginning:
-- Commit to migrating from galaxy.ini to galaxy.yml (if not done previously)
Even though its possible to stick with the old galaxy.ini file, it seems best
to move to the new format - my feeling is, probably the migration will have to
happen at some point anyway, and also the documentation is more focussed on the
newer galaxy.yml version.
-- Decide on how you're going to handle job submission (essentially, how you're
going to replace the old job handler processes).
There are a number of possibilities outlined in the documentation here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#deployment-options
It seems important to figure this out at the beginning because it impacts a lot of the
configuration specifics that follow. For our setup case "uwsgi + mules" looked
like the most appropriate:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi-for-web-serving-with-mules-as-job-handlers
Then once you're ready to start (nb backing up the database etc before starting
are implicitly assumed!):
-- Update the Galaxy source code
The standard recipe for this seems to work okay i.e.
git fetch origin && git checkout release_18.05 && git pull --ff-only origin
release_18.05
(You'll probably also need to run scripts/common_startup.sh to update the
Python packages, and scripts/manage_db.py to update the database schema.)
-- Create galaxy.yml and configure the "galaxy" section:
You can make a galaxy.yml from the sample version and go through the "galaxy" section
replicating the settings from galaxy.ini. (The settings in the "uwsgi" section can wait
at this stage.) Don't forget to move galaxy.ini out of the way once you're done.
-- Configure the "uwsgi" section of galaxy.yml:
The options for various scenarios are given in the "Deployment/uwsgi" section:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi
but the required settings seem to be spread out between different sections so
(for me) it felt like it took some piecing together to get it to work when I
first tried this.
(Also I was thrown a little by the "socket" parameter not having a placeholder
in sample galaxy.yml.)
The core uwsgi options are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#configuration-common-to-all-uwsgi-deployment-styles
then you need to set other options dependent on:
1. how job submission is handled (e.g. options for the "uwsgi + mules" setup in my case)
2. how you're proxying Galaxy (for Apache or nginx it looks like it's just setting the uwsgi
"socket" option, unless you're serving Galaxy from a URL prefix in which case there are a
few more options needed - these are covered in the documentation.)
-- Fix your job_conf.xml file
If you have an existing job_conf.xml file in your config directory then you might need to
update that too (specifically the "handlers" section) depending on the job
submission handling you've decided to use.
-- Use supervisor to manage your uwsgi processes
The docs for this are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#starting-and-stopping
Essentially supervisor replaces "run.sh", so this becomes how you start and stop Galaxy (e.g.
"supervisorctl start galaxy:" replaces "GALAXY_RUN_ALL=1 sh run.sh --daemon".)
I think this is quite straightforward from the docs but again you need to tailor it for
how you're handling job submission (e.g the "uwsgi + mules" doesn't need any
handlers defining, but it looks like other strategies needs extra stuff defining here).
-- Set up the proxying
Assuming you're proxing via either Apache or nginx (we're using nginx), these
are covered in the relevant sections:
https://docs.galaxyproject.org/en/release_18.05/admin/apache.html
https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html
The main difference I noticed for nginx is that you no longer need to use the
"upstream"/"proxy_pass" directives, as nginx can talk directly to uwsgi using a
"uwsgi_pass" directive that connects to the socket defined in the "uwsgi" section of galaxy.yml
previously.
(There's also a "gotcha" if you're nginx configuration also tries to serve
"static/scripts/packed", which needs to be removed - this is noted in the "breaking
changes" for the 18.05 docs but is easy to miss.)
Hopefully this is some help. There's lots of documentation, which is a good
thing but which can sometimes make it harder to see where to start.
Good luck with the update,
Best wishes
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core Facility
University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
________________________________________
From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of
Jochen Bick [jochen.b...@usys.ethz.ch]
Sent: Monday, July 09, 2018 12:49 PM
To: galaxy-dev@lists.galaxyproject.org
Subject: [galaxy-dev] From Paste to uWSGI
Hi all,
we would like to update to 18.05 and thereby also update our Paste Server.
https://docs.galaxyproject.org/en/latest/admin/scaling.html
Is there any tutorial on how to update from Paste to uWSGI?
Cheers Jochen
--
ETH Zurich
*Jochen Bick*
Animal Physiology
Institute of Agricultural Sciences
Postal address: Universitätstrasse 2 / LFW B 58.1
8092 Zurich, Switzerland
Office: Eschikon 27
8315 Lindau, Switzerland
Phone +41 52 354 92 06
jochen.b...@usys.ethz.ch <mailto:jochen.b...@usys.ethz.ch> www.ap.ethz.ch
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this and other Galaxy
lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
------------------------------
Message: 3
Date: Mon, 9 Jul 2018 11:36:27 -0400
From: Martin Čech <mar...@bx.psu.edu>
To: Peter Briggs <peter.bri...@manchester.ac.uk>
Cc: "galaxy-dev@lists.galaxyproject.org"
<galaxy-dev@lists.galaxyproject.org>
Subject: Re: [galaxy-dev] From Paste to uWSGI
Message-ID:
<CAHKBquGXMTog-MUgLbPWD2wgmXvpOxiQGRa4+QCWk5EAHGK3=g...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Another recently updated resource for uwsgi is the training slides:
https://galaxyproject.github.io/dagobah-training/2018-gccbosc/10-uwsgi/uwsgi.html#1
M.
On Mon, Jul 9, 2018 at 11:31 AM Peter Briggs <peter.bri...@manchester.ac.uk>
wrote:
Hello Jochen
I'm not aware of a tutorial but here are some observations from my
(very) recent experiences upgrading a local paste-based 17.09 instance
to a uwsgi-based 18.05 configuration. It's not comprehensive because
the configuration will depend on decisions you need to make based on
your local setup, but it might be helpful as a starting point.
Before beginning:
-- Commit to migrating from galaxy.ini to galaxy.yml (if not done
previously)
Even though its possible to stick with the old galaxy.ini file, it
seems best to move to the new format - my feeling is, probably the
migration will have to happen at some point anyway, and also the
documentation is more focussed on the newer galaxy.yml version.
-- Decide on how you're going to handle job submission (essentially,
how you're going to replace the old job handler processes).
There are a number of possibilities outlined in the documentation here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#dep
loyment-options
It seems important to figure this out at the beginning because it
impacts a lot of the configuration specifics that follow. For our
setup case "uwsgi
+ mules" looked like the most appropriate:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uws
gi-for-web-serving-with-mules-as-job-handlers
Then once you're ready to start (nb backing up the database etc before
starting are implicitly assumed!):
-- Update the Galaxy source code
The standard recipe for this seems to work okay i.e.
git fetch origin && git checkout release_18.05 && git pull --ff-only
origin release_18.05
(You'll probably also need to run scripts/common_startup.sh to update
the Python packages, and scripts/manage_db.py to update the database
schema.)
-- Create galaxy.yml and configure the "galaxy" section:
You can make a galaxy.yml from the sample version and go through the
"galaxy" section replicating the settings from galaxy.ini. (The
settings in the "uwsgi" section can wait at this stage.) Don't forget
to move galaxy.ini out of the way once you're done.
-- Configure the "uwsgi" section of galaxy.yml:
The options for various scenarios are given in the "Deployment/uwsgi"
section:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uws
gi
but the required settings seem to be spread out between different
sections so (for me) it felt like it took some piecing together to get
it to work when I first tried this.
(Also I was thrown a little by the "socket" parameter not having a
placeholder in sample galaxy.yml.)
The core uwsgi options are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#con
figuration-common-to-all-uwsgi-deployment-styles
then you need to set other options dependent on:
1. how job submission is handled (e.g. options for the "uwsgi + mules"
setup in my case)
2. how you're proxying Galaxy (for Apache or nginx it looks like it's
just setting the uwsgi "socket" option, unless you're serving Galaxy
from a URL prefix in which case there are a few more options needed -
these are covered in the documentation.)
-- Fix your job_conf.xml file
If you have an existing job_conf.xml file in your config directory
then you might need to update that too (specifically the "handlers"
section) depending on the job submission handling you've decided to use.
-- Use supervisor to manage your uwsgi processes
The docs for this are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#sta
rting-and-stopping
Essentially supervisor replaces "run.sh", so this becomes how you
start and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces
"GALAXY_RUN_ALL=1 sh run.sh --daemon".)
I think this is quite straightforward from the docs but again you need
to tailor it for how you're handling job submission (e.g the "uwsgi + mules"
doesn't need any handlers defining, but it looks like other strategies
needs extra stuff defining here).
-- Set up the proxying
Assuming you're proxing via either Apache or nginx (we're using
nginx), these are covered in the relevant sections:
https://docs.galaxyproject.org/en/release_18.05/admin/apache.html
https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html
The main difference I noticed for nginx is that you no longer need to
use the "upstream"/"proxy_pass" directives, as nginx can talk directly
to uwsgi using a "uwsgi_pass" directive that connects to the socket
defined in the "uwsgi" section of galaxy.yml previously.
(There's also a "gotcha" if you're nginx configuration also tries to
serve "static/scripts/packed", which needs to be removed - this is
noted in the "breaking changes" for the 18.05 docs but is easy to
miss.)
Hopefully this is some help. There's lots of documentation, which is a
good thing but which can sometimes make it harder to see where to start.
Good luck with the update,
Best wishes
Peter
--
Peter Briggs peter.bri...@manchester.ac.uk Bioinformatics Core
Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
________________________________________
From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on
behalf of Jochen Bick [jochen.b...@usys.ethz.ch]
Sent: Monday, July 09, 2018 12:49 PM
To: galaxy-dev@lists.galaxyproject.org
Subject: [galaxy-dev] From Paste to uWSGI
Hi all,
we would like to update to 18.05 and thereby also update our Paste Server.
https://docs.galaxyproject.org/en/latest/admin/scaling.html
Is there any tutorial on how to update from Paste to uWSGI?
Cheers Jochen
--
ETH Zurich
*Jochen Bick*
Animal Physiology
Institute of Agricultural Sciences
Postal address: Universitätstrasse 2
<https://maps.google.com/?q=Universit%C3%A4tstrasse+2&entry=gmail&sour
ce=g>
/ LFW B 58.1
8092 Zurich, Switzerland
Office: Eschikon 27
8315 Lindau, Switzerland
Phone +41 52 354 92 06 <+41%2052%20354%2092%2006>
jochen.b...@usys.ethz.ch <mailto:jochen.b...@usys.ethz.ch>
www.ap.ethz.ch
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this and other
Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this and other
Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20180709/a7d44c29/attachment-0001.html>
------------------------------
Subject: Digest Footer
_______________________________________________
galaxy-dev mailing list
galaxy-dev@lists.galaxyproject.org
https://lists.galaxyproject.org/listinfo/galaxy-dev
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
------------------------------
End of galaxy-dev Digest, Vol 145, Issue 3
******************************************
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
