[galaxy-dev] Galaxy migration to docker environment
Hi all, We’re looking to migrate our Galaxy instance from a VM based solution to a docker based solution. Therefore, we need to move all our user and reference data. What is the best possible solution to migrate all data and change the path of said data in the database? Thanks! M ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Building tool dependencies
Hi All! Thanks for chiming in! I’ll give the conda recipe approach a go, this seems to be the easiest way to go, and the most futureproof! Thanks! M > Message: 1 > Date: Tue, 5 Jul 2016 06:56:00 + > From: Matthias De Smet <matthias.des...@ugent.be> > To: "galaxy-dev@lists.galaxyproject.org" > <galaxy-dev@lists.galaxyproject.org> > Subject: Re: [galaxy-dev] Building tool dependencies > Message-ID: <d70eed58-f05e-4084-9323-1b78c707b...@ugent.be> > Content-Type: text/plain; charset="utf-8" > > >> >> Hi >> >> I’m trying to put together a tool + dependency package to use umi-tools >> (https://github.com/CGATOxford/UMI-tools) from Galaxy. >> I’m at a loss on how I should organise this. >> >> The package is available from either github or pip. Which one is preferable >> to use? How do I make sure the package dependencies are installed? I’ve >> found the and installing it with > type="shell_command">python setup.py install. This, however, tries >> to install the package to /usr/bin/ which isn’t desirable (obviously). >> >> Thanks a lot! >> M >> > > > -- > > Message: 2 > Date: Tue, 5 Jul 2016 09:33:09 +0200 > From: Peter van Heusden <p...@sanbi.ac.za> > To: galaxy-...@lists.bx.psu.edu > Subject: Re: [galaxy-dev] Building tool dependencies > Message-ID: > <cak1rexih9jxq19gysxdkdr6ync9azgabkfzxaktfm0m+tc2...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Actually, for recent Galaxy, I'd recommend making a conda recipe for > UMI-tools and contributing it to bioconda: > > https://bioconda.github.io/ > > Then your tool can just depend on this and the conda dependency resolver > (default in Galaxy since 16.01) can find it. Others can comment on the > recent work on conda/toolshed integration. > > Peter > > On 5 July 2016 at 08:56, Matthias De Smet <matthias.des...@ugent.be> wrote: > >> >>> >>> Hi >>> >>> I’m trying to put together a tool + dependency package to use umi-tools ( >> https://github.com/CGATOxford/UMI-tools) from Galaxy. >>> I’m at a loss on how I should organise this. >>> >>> The package is available from either github or pip. Which one is >> preferable to use? How do I make sure the package dependencies are >> installed? I’ve found the and installing it with > type="shell_command">python setup.py install. This, however, tries >> to install the package to /usr/bin/ which isn’t desirable (obviously). >>> >>> Thanks a lot! >>> M >>> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > -- next part -- > An HTML attachment was scrubbed... > URL: > <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20160705/0b422d9d/attachment-0001.html> > > -- > > Message: 3 > Date: Tue, 5 Jul 2016 08:49:54 +0100 > From: Peter Briggs <peter.bri...@manchester.ac.uk> > To: galaxy-dev@lists.galaxyproject.org > Subject: Re: [galaxy-dev] Building tool dependencies > Message-ID: <577b66a2.60...@manchester.ac.uk> > Content-Type: text/plain; charset=utf-8; format=flowed > > Hello Matthias > > From what I've seen there are a number of different recipes used for > installing Python packages via the toolshed; the approach you're using > seems to be pretty common. In these cases the usual 'shell_command' > seems to be: > > python setup.py install --install-lib $INSTALL_DIR/lib/python > --install-scripts $INSTALL_DIR/bin > > which should avoid installing to /usr/bin. > > You'll then need to do something like: > > >name="PYTHONPATH">$INSTALL_DIR/lib/python >name="PATH">$INSTALL_DIR/bin > ... > > > to ensure that the correct environment is set up when the tool is used. > > Aside from this: elsewhere I've seen other approaches including a > "magic" "package" command where the whole download and install is taken > care of in a single tag. For example: > > ... > > > > > sha256sum="e01853dfe111f3aea005315573400b72
Re: [galaxy-dev] Building tool dependencies
> > Hi > > I’m trying to put together a tool + dependency package to use umi-tools > (https://github.com/CGATOxford/UMI-tools) from Galaxy. > I’m at a loss on how I should organise this. > > The package is available from either github or pip. Which one is preferable > to use? How do I make sure the package dependencies are installed? I’ve > found the and installing it with type="shell_command">python setup.py install. This, however, tries > to install the package to /usr/bin/ which isn’t desirable (obviously). > > Thanks a lot! > M > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy broken after update to 16.01
Hi all,
We were tracking the galaxy master branch on our local server and are looking
to migrate to v16.01.
The galaxy processes are managed bij supervisor as shown here (
https://wiki.galaxyproject.org/Admin/Config/Performance/Scaling).
After updating all dependencies in our venv we tried to restart the server but
got the following issue:
Loading paste environment: config:/home/galaxy/galaxy-dist/config/galaxy.ini
Traceback (most recent call last):
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py",
line 247, in loadapp
return loadobj(APP, uri, name=name, **kw)
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py",
line 271, in loadobj
global_conf=global_conf)
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py",
line 296, in loadcontext
global_conf=global_conf)
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py",
line 320, in _loadconfig
return loader.get_context(object_type, name, global_conf)
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py",
line 458, in get_context
section)
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py",
line 517, in _context_from_explicit
value = import_string(found_expr)
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py",
line 22, in import_string
return pkg_resources.EntryPoint.parse("x=" + s).load(False)
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 2380, in load
return self.resolve()
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 2386, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "lib/galaxy/web/buildapp.py", line 5, in
from galaxy.webapps.galaxy.buildapp import app_factory
File "lib/galaxy/webapps/galaxy/buildapp.py", line 15, in
import galaxy.app
File "lib/galaxy/app.py", line 9, in
import galaxy.queues
File "lib/galaxy/queues.py", line 9, in
from kombu import Exchange, Queue, Connection
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/__init__.py",
line 67, in __getattr__
module = __import__(object_origins[name], None, None, [name])
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/entity.py",
line 10, in
from .abstract import MaybeChannelBound, Object
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/abstract.py",
line 12, in
from .connection import maybe_channel
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/connection.py",
line 19, in
from kombu import exceptions
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/exceptions.py",
line 12, in
from amqp import ChannelError, ConnectionError, ResourceError
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/amqp/__init__.py",
line 34, in
from .connection import Connection # noqa
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/amqp/connection.py",
line 40, in
from .transport import create_transport
File
"/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/amqp/transport.py",
line 21, in
import ssl
File "/Shared/Software/Cellar/python/2.7.11/lib/python2.7/ssl.py", line 97,
in
import _ssl # if we can't import it, let the error propagate
ImportError: /home/galaxy/galaxy-dist/.venv/lib/python2.7/lib-dynload/_ssl.so:
undefined symbol: SSL_CTX_set_alpn_protos
So far we have tried:
- rebuilding the venv
- reinstalling python
Funny thing is, when we start the configuration with the “run.sh” script
instead of through supervisor everything seems to work just fine.
We’re at a loss, please help!
Thanks!
Matthias
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Filtering dynamic options in tool with tagset
Hi All, I’m trying to implement a tool that get a list of options through the “dynamic_options” tag. I try to filter the options that get returned with another param, but it doensn’t seem to work. my tool xml looks like this from the archive to a data library test.sh > $log Anyone with experience implementing filters? (Gx 15.10 on Ubuntu 14.04.3) Thanks! Matthias ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Filtering dynamic options in tool with tagset
Hi Hans-Rudolf Thanks again, you’re a lifesaver! Any suggestions on how I could eliminate the tag, as the wiki states it’s deprecated? I checked out but don’t see a straightforward wau to get the same results. Matthias > Op 27 jan. 2016, om 10:20 heeft Hans-Rudolf Hotz <h...@fmi.ch> het volgende > geschreven: > > Hi Matthias > > I don't think you can do that...I would delegate the filtering to the > 'list_files' function, i.e. by calling: > > list_files(archive_source,query) > > > Regards, Hans-Rudolf > > > On 01/27/2016 10:07 AM, Matthias De Smet wrote: >> >> Hi All, >> >> I’m trying to implement a tool that get a list of options through the >> “dynamic_options” tag. I try to filter the options that get returned with >> another param, but it doensn’t seem to work. >> >> my tool xml looks like this >> >> >> from the archive to a data library >> >> >> test.sh > $log >> >> >> >> >> >> >> >> >> >> > dynamic_options="list_files(archive_source)"> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Anyone with experience implementing filters? (Gx 15.10 on Ubuntu 14.04.3) >> >> Thanks! >> Matthias >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Filtering dynamic options in tool with tagset
So it seems it has been deprecated a bit prematurely. Maybe the Galaxy team should “undeprecate” it untill a viable alternative is found. Anyway, thanks again! Matthias > Op 27 jan. 2016, om 10:33 heeft Hans-Rudolf Hotz <h...@fmi.ch> het volgende > geschreven: > > > > On 01/27/2016 10:29 AM, Matthias De Smet wrote: >> Hi Hans-Rudolf >> >> Thanks again, you’re a lifesaver! >> Any suggestions on how I could eliminate the tag, as the wiki states >> it’s deprecated? > > well, it has been 'deprecated' for many years now. > > >> I checked out but don’t see a straightforward wau to get the same >> results. > > same for me :( > > > Hans-Rudolf > > > > >> >> Matthias >> >>> Op 27 jan. 2016, om 10:20 heeft Hans-Rudolf Hotz <h...@fmi.ch> het volgende >>> geschreven: >>> >>> Hi Matthias >>> >>> I don't think you can do that...I would delegate the filtering to the >>> 'list_files' function, i.e. by calling: >>> >>> list_files(archive_source,query) >>> >>> >>> Regards, Hans-Rudolf >>> >>> >>> On 01/27/2016 10:07 AM, Matthias De Smet wrote: >>>> >>>> Hi All, >>>> >>>> I’m trying to implement a tool that get a list of options through the >>>> “dynamic_options” tag. I try to filter the options that get returned with >>>> another param, but it doensn’t seem to work. >>>> >>>> my tool xml looks like this >>>> >>>> >>>> from the archive to a data library >>>> >>>> >>>> test.sh > $log >>>> >>>> >>>> >>> label="Choose source archive"> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> dynamic_options="list_files(archive_source)"> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Anyone with experience implementing filters? (Gx 15.10 on Ubuntu 14.04.3) >>>> >>>> Thanks! >>>> Matthias >>>> ___ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> https://lists.galaxyproject.org/ >>>> >>>> To search Galaxy mailing lists use the unified search at: >>>> http://galaxyproject.org/search/mailinglists/ >>>> >> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] access system variables in tool xml
Hi again I’m trying to create a tool that interacts with galaxy through the API. For this, I need to get the users API key (or equivalent) en the domain name of the Galaxy host. The purpose of this tool is to get data and put it in a data library, so I suppose I need to use the API. Or is there another way to do this with a tool? Thanks! Matthias ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Python ValueError: too many values to unpack
Wow, great tip! This fixed it!
Thanks a bunch!
Matthias
> Op 26 jan. 2016, om 15:20 heeft Hans-Rudolf Hotz <h...@fmi.ch> het volgende
> geschreven:
>
> Hi Matthias
>
>
> Make sure 'listing' is a list of (displayName,value,boolean)
>
> Though, there might be other errors as well
>
>
> Regards, Hans-Rudolf
>
> I am not sure, whether this
> On 01/26/2016 01:22 PM, Matthias De Smet wrote:
>> Hi all,
>>
>> I’m trying to develop a tool that lets you select a file on the server.
>> I’m using the dynamic_options setting to let Galaxy render a dropdown
>> list of available file.
>>
>>
>> However, when I try to test the tool in its current form, I get the
>> following error:
>>
>>
>> galaxy.web.framework.decorators ERROR 2016-01-26 13:08:35,649 Uncaught
>> exception in exposed API method:
>> Traceback (most recent call last):
>> File "lib/galaxy/web/framework/decorators.py", line 260, in decorator
>> rval = func( self, trans, *args, **kwargs)
>> File "lib/galaxy/webapps/galaxy/api/tools.py", line 98, in build
>> return tool.to_json(trans, kwd.get('inputs', kwd))
>> File "lib/galaxy/tools/__init__.py", line 2602, in to_json
>> populate_state(trans, self.inputs, state_inputs, state_errors,
>> params.__dict__)
>> File "lib/galaxy/tools/__init__.py", line 2471, in populate_state
>> state[input.name] = input.get_initial_value(trans, context,
>> history=history)
>> File "lib/galaxy/tools/parameters/basic.py", line 984, in
>> get_initial_value
>> value = [ optval for _, optval, selected in options if selected ]
>> ValueError: too many values to unpack
>>
>>
>> My tool xml looks like this:
>>
>>
>>
>>
>> from the archive to a data library
>>
>>
>> test.sh > $log
>>
>>
>> > label="Choose source archive">
>>
>>
>>
>>
>>
>>
>> > dynamic_options="list_files(archive_source,query)"/>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> This tool copies data from and archive folder to your history. Keep in
>> mind this folder MUST be readable by Galaxy in order for this tool to work.
>>
>>
>>
>>
>>
>> and the script called in the “code” tag looks like this
>>
>>
>> def list_files(dir,query): listing=[]
>> for fname in listdir(dir):
>> fullpath = path.join(dir, fname) #only select datafiles, not md5
>> checksums
>> if path.isfile(fullpath) and not fullpath.endswith(".md5"):
>> listing.append( fname )
>> return listing
>>
>>
>> This script should return a list with filenames that should be displayed
>> in the dropdown list.
>>
>> I’m tracking Galaxy release 15.10, running on a Ubuntu 14.04.3 server
>>
>> Thanks for your help!
>>
>> Matthias
>>
>>
>>
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client. To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>> http://galaxyproject.org/search/mailinglists/
>>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool development
Hi Hans-Rudolf Yes, we have considered using data libraries, but for this particular use case we would prefer an approach like this. Our users are notoriously bad at data management and we would like to limit the amount of data that gets loaded in Galaxy. The networked share does not store all data permanently, so we need something to query what is available so the users can load that into their history. Ideally the datasets also shouldn’t be copied into Galaxy, but symlinked, if possible. Thanks for the tip on ‘dynamic_options’. Hadn’t noticed it before. Matthias > Op 19 jan. 2016, om 13:40 heeft Hans-Rudolf Hotz <h...@fmi.ch> het volgende > geschreven: > > Hi Matthias > > Have you considered "Data Libraries"? > > A cron job can query your LIMS and use the API to create "Data Libraries". > Then the user can select which datasets she/he wants to import. > > > And back to your original question: Yes, it is possible to create interactive > tools. Check out the 'dynamic_options' examples on > > https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax > > > Regards, Hans-Rudolf > > On 01/19/2016 12:18 PM, Matthias De Smet wrote: >> Hi All, >> >> I would like to create a tool to import data from a sequencer on a networked >> share to a users history. >> The ideal setup would be that a query can be submitted to a database which >> contains all metadata from the sequencer + the path of the sequencer output. >> The tool should then show the available datasets found in the database and >> provide checkboxes to select which data should be imported. >> >> My question is: how do you guys propose I get started on this? Do I use the >> API? Is it even possible to get a tool to return some output without >> submitting it as a job, e.g. using it interactively? >> >> Thanks for the help! >> >> PS: I know the basics of tool development, but I haven’t got much >> experience, so all comments are welcome! >> >> Matthias >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
