Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi,

Am 06.08.2015 um 15:19 schrieb Tiago Rodrigues Antao:
> On Thu, 06 Aug 2015 14:00:02 +0200
> Björn Grüning  wrote:
> 
> 
>> the first one is the one we are currently working on. It's an
>> interface to create your own Galaxy Docker flavours. Drag & Drop your
>> tools data managers and so on ... click build and either get a Docker
>> Image or a Docker file.
>>
>> But everything is in flux at the moment. So please jump in and
>> contribute your ideas wherever you can!
> 
> 
> 1. A notion of a bundle. Say for example a RNA-Seq bundle with packages
> for that kind of analysis (which the user could still add/remove)

We call it flavours:
https://raw.githubusercontent.com/bgruening/docker-galaxy-stable/master/chart.png

And yes this should be possible.

> 2. The ability to define some kind of iteration with a cluster: choose
> cluster type (slurm, SGE, LSF, ...), for example

This could be as easy as changing two files, so yes should be possible!

Ciao,
Bjoern

> Tiago
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

On Thu, 06 Aug 2015 14:00:02 +0200
Björn Grüning  wrote:


> the first one is the one we are currently working on. It's an
> interface to create your own Galaxy Docker flavours. Drag & Drop your
> tools data managers and so on ... click build and either get a Docker
> Image or a Docker file.
> 
> But everything is in flux at the moment. So please jump in and
> contribute your ideas wherever you can!


1. A notion of a bundle. Say for example a RNA-Seq bundle with packages
for that kind of analysis (which the user could still add/remove)

2. The ability to define some kind of iteration with a cluster: choose
cluster type (slurm, SGE, LSF, ...), for example

Tiago
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi,

the first one is the one we are currently working on. It's an interface
to create your own Galaxy Docker flavours. Drag & Drop your tools data
managers and so on ... click build and either get a Docker Image or a
Docker file.

But everything is in flux at the moment. So please jump in and
contribute your ideas wherever you can!

Bjoern

Am 06.08.2015 um 13:42 schrieb Tiago Rodrigues Antao:
> On Wed, 5 Aug 2015 14:10:57 -0500
> Eric Rasche  wrote:
> 
>>- http://wookoouk.github.io/galaxyFlavorGenerator/
>>- https://github.com/galaxyFlavorsGenerators/galaxyFlavorGenerator/
> 
> 
> The first link works (for Docker), but the second has only a (empty)
> readme file...
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

On Wed, 5 Aug 2015 14:10:57 -0500
Eric Rasche  wrote:

>- http://wookoouk.github.io/galaxyFlavorGenerator/
>- https://github.com/galaxyFlavorsGenerators/galaxyFlavorGenerator/


The first link works (for Docker), but the second has only a (empty)
readme file...
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Eric Rasche]

Tiago,

Some work is already in progress for that. Please see

   - http://wookoouk.github.io/galaxyFlavorGenerator/
   - https://github.com/galaxyFlavorsGenerators/galaxyFlavorGenerator/

I don't believe it handles configuration of the cluster backend, but I'm
sure that would be a good addition to the project.

Ciao,
Eric

2015-08-05 10:02 GMT-05:00 Tiago Rodrigues Antao :

> On Tue, 4 Aug 2015 17:26:00 +
> "Kandalaft, Iyad"  wrote:
>
> > As a supplement to Oksana's original inquiry, we, at AAFC, couple
> > Galaxy to SGE  and possibly a different grid engine in the future for
> > job scheduling.  The old-fashioned way of doing that is to unify
> > storage path and UID/GID between the application server and cluster.
>
>
> I am starting to warm of the idea of making a small web interface that
> asks for a few configuration options:
> 1. type of grid engine
> 2. desired docker/galaxy flavour (RNA, exome, WGS, ...)
> 3. ... (e.g. intial admin user and password)
>
> and would pull Björn's relevant Dockerfile, patch it and generate a new
> one, tailored to the need of the user.
>
> I think it is impossible to have a set of dockerfiles for all the
> desired configurations (think analysis flavours times number of cluster
> types - for example).
>
> I will need to implement something like this very soon, and I would not
> mind to do it in a way that the community could reuse...
>
> So comments would be appreciated...
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>



-- 
Eric Rasche
Programmer II

Center for Phage Technology
Rm 312A, BioBio
Texas A&M University
College Station, TX 77843
404-692-2048
e...@tamu.edu
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

On Tue, 4 Aug 2015 17:26:00 +
"Kandalaft, Iyad"  wrote:

> As a supplement to Oksana's original inquiry, we, at AAFC, couple
> Galaxy to SGE  and possibly a different grid engine in the future for
> job scheduling.  The old-fashioned way of doing that is to unify
> storage path and UID/GID between the application server and cluster.


I am starting to warm of the idea of making a small web interface that
asks for a few configuration options:
1. type of grid engine
2. desired docker/galaxy flavour (RNA, exome, WGS, ...)
3. ... (e.g. intial admin user and password)

and would pull Björn's relevant Dockerfile, patch it and generate a new
one, tailored to the need of the user.

I think it is impossible to have a set of dockerfiles for all the
desired configurations (think analysis flavours times number of cluster
types - for example).

I will need to implement something like this very soon, and I would not
mind to do it in a way that the community could reuse...

So comments would be appreciated...
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Thanks!

@Brian please add your ideas and use cases so we can figure out a way
how to support this :)

Thanks,
Bjoern

Am 05.08.2015 um 16:26 schrieb Oksana Korol:
> Created issue #74, with reference to this discussion.
> 
> Cheers,
> Oksana
> 
> On Wed, Aug 5, 2015 at 9:52 AM, Björn Grüning 
> wrote:
> 
>> Hi Brian, Oksana,
>>
>> let's move this discussion to github and try to get something done for
>> the next+1 release.
>>
>> 15.07/15.08 or what ever it will be is really close so I would like to
>> not rush here. But as soon as we have release this version. We can try
>> to get something in :dev that addresses these needs.
>>
>> What do you think? Can you create an Issue for it?
>> Thanks!
>> Bjoern
>>
>> Am 05.08.2015 um 15:39 schrieb Oksana Korol:
>>> HI Brian,
>>>
>>> Thanks for sharing your code. We have just discussed your approach as one
>>> possibility of solving our problem, but delayed it as it involved too
>> much
>>> hacking :). Now that you've done the hacking, we may revisit it.
>>> We were also discussing a possibility of creating access groups with
>> Galaxy
>>> UID and cluster process ID to get around the permissions problem that
>> Iyad
>>> has described. This would involve extra setup outside Galaxy container,
>> but
>>> would possibly remove the need to change GID and UID. Will reply when
>> we've
>>> solidified and tested our approach.
>>>
>>> Bjorn,
>>>
>>> I would second Brian on being able to configure Galaxy for different
>>> environments. To us that was the primary reason of going the docker way.
>> We
>>> have considered vagrant, since it seemed like a more mature and more
>>> flexible technology for running in production, but went with docker,
>>> because Galaxy community supports it.
>>>
>>> I would personally like to be able to change build-time variables at
>> build,
>>> since it's more logical, requires less hacks and the build speed is not
>> as
>>> important to me when building containers for different environments,
>> since
>>> I'm not going to do that often. I.e.I would still have your lightweight
>> way
>>> as a default, but allow to change it for those who need it. That said, I
>>> would welcome any standardized solution, be it run time or build time.
>>>
>>> Cheers,
>>> Oksana
>>>
>>> On Wed, Aug 5, 2015 at 4:30 AM, Björn Grüning >>
>>> wrote:
>>>
 Hi Brian,

 thanks for you mail! Very appreciated, please see my comments inline.

>> We only have one cluster, so we will have both test and production
>> environments running on it. I understand that at a minimum we'd need
>> different Galaxy home dir and UID/GID.
>
>
> For our local deployment of Galaxy with Docker, which was inspired by a
> very early version of Björn's docker-galaxy-stable, we use an
>> entrypoint
> script to remap the Galaxy user's UID and GID inside the container from
> environment variables GALAXY_UID and GALAXY_GID passed to `docker run`:

 Is this really needed. We could do this in the main image as well, but
 currently I haven't seen a compelling usecase for this. The downside is
 that we need to do a lot of `chown/chmod` magic during container
>> startup.
 But I'm totally willing to do so, if this is was the community is
>> needed.

 Please also see this latest PR:
 https://github.com/bgruening/docker-galaxy-stable/pull/71

>

>> https://github.com/fredhutchio/docker-galaxy/blob/aa07c5f684d55499a87e77c62aceda707b63077f/docker-entrypoint.sh#L30
>
> Just below that, starting at line 42, is an admittedly hacky way of
> relocating the Galaxy directory hierarchy (exported data and/or code)
 from
> its container default location /galaxy to another location specified by
 the
> GALAXY_ROOT environment variable. It essentially turns the container
>> into
> an installer when `--reroot` or `--upgrade` is passed to the entrypoint
> script. `--reroot` copies the container's entire /galaxy hierarchy to
> $GALAXY_ROOT for initializing an instance from the container.
>> `--upgrade`
> copies just the container's Galaxy distribution in /galaxy/stable to
> $GALAXY_ROOT/stable (aka $GALAXY_HOME) for upgrading an existing
>> instance
> from a more recent image.

 Interesting!
 At least for easier upgrading we took a slightly different approach,
 described here:


>> https://github.com/bgruening/docker-galaxy-stable/tree/dev#upgrading-images

 Also we store nowadays many original config files in /etc/galaxy/ to
 make the upgrading smoother.
 I would be interested in your opinion and if these latest developments
 would solve your use-cases.

> It's definitely nice being able to configure things like this at
>> runtime
> for e.g. differentiating between test and production! Would
>> functionality
> like this be useful/welcome in docker-galaxy-stable?

 I'm all for it, if this is what is needed by the comm

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Oksana Korol]

Created issue #74, with reference to this discussion.

Cheers,
Oksana

On Wed, Aug 5, 2015 at 9:52 AM, Björn Grüning 
wrote:

> Hi Brian, Oksana,
>
> let's move this discussion to github and try to get something done for
> the next+1 release.
>
> 15.07/15.08 or what ever it will be is really close so I would like to
> not rush here. But as soon as we have release this version. We can try
> to get something in :dev that addresses these needs.
>
> What do you think? Can you create an Issue for it?
> Thanks!
> Bjoern
>
> Am 05.08.2015 um 15:39 schrieb Oksana Korol:
> > HI Brian,
> >
> > Thanks for sharing your code. We have just discussed your approach as one
> > possibility of solving our problem, but delayed it as it involved too
> much
> > hacking :). Now that you've done the hacking, we may revisit it.
> > We were also discussing a possibility of creating access groups with
> Galaxy
> > UID and cluster process ID to get around the permissions problem that
> Iyad
> > has described. This would involve extra setup outside Galaxy container,
> but
> > would possibly remove the need to change GID and UID. Will reply when
> we've
> > solidified and tested our approach.
> >
> > Bjorn,
> >
> > I would second Brian on being able to configure Galaxy for different
> > environments. To us that was the primary reason of going the docker way.
> We
> > have considered vagrant, since it seemed like a more mature and more
> > flexible technology for running in production, but went with docker,
> > because Galaxy community supports it.
> >
> > I would personally like to be able to change build-time variables at
> build,
> > since it's more logical, requires less hacks and the build speed is not
> as
> > important to me when building containers for different environments,
> since
> > I'm not going to do that often. I.e.I would still have your lightweight
> way
> > as a default, but allow to change it for those who need it. That said, I
> > would welcome any standardized solution, be it run time or build time.
> >
> > Cheers,
> > Oksana
> >
> > On Wed, Aug 5, 2015 at 4:30 AM, Björn Grüning  >
> > wrote:
> >
> >> Hi Brian,
> >>
> >> thanks for you mail! Very appreciated, please see my comments inline.
> >>
>  We only have one cluster, so we will have both test and production
>  environments running on it. I understand that at a minimum we'd need
>  different Galaxy home dir and UID/GID.
> >>>
> >>>
> >>> For our local deployment of Galaxy with Docker, which was inspired by a
> >>> very early version of Björn's docker-galaxy-stable, we use an
> entrypoint
> >>> script to remap the Galaxy user's UID and GID inside the container from
> >>> environment variables GALAXY_UID and GALAXY_GID passed to `docker run`:
> >>
> >> Is this really needed. We could do this in the main image as well, but
> >> currently I haven't seen a compelling usecase for this. The downside is
> >> that we need to do a lot of `chown/chmod` magic during container
> startup.
> >> But I'm totally willing to do so, if this is was the community is
> needed.
> >>
> >> Please also see this latest PR:
> >> https://github.com/bgruening/docker-galaxy-stable/pull/71
> >>
> >>>
> >>
> https://github.com/fredhutchio/docker-galaxy/blob/aa07c5f684d55499a87e77c62aceda707b63077f/docker-entrypoint.sh#L30
> >>>
> >>> Just below that, starting at line 42, is an admittedly hacky way of
> >>> relocating the Galaxy directory hierarchy (exported data and/or code)
> >> from
> >>> its container default location /galaxy to another location specified by
> >> the
> >>> GALAXY_ROOT environment variable. It essentially turns the container
> into
> >>> an installer when `--reroot` or `--upgrade` is passed to the entrypoint
> >>> script. `--reroot` copies the container's entire /galaxy hierarchy to
> >>> $GALAXY_ROOT for initializing an instance from the container.
> `--upgrade`
> >>> copies just the container's Galaxy distribution in /galaxy/stable to
> >>> $GALAXY_ROOT/stable (aka $GALAXY_HOME) for upgrading an existing
> instance
> >>> from a more recent image.
> >>
> >> Interesting!
> >> At least for easier upgrading we took a slightly different approach,
> >> described here:
> >>
> >>
> https://github.com/bgruening/docker-galaxy-stable/tree/dev#upgrading-images
> >>
> >> Also we store nowadays many original config files in /etc/galaxy/ to
> >> make the upgrading smoother.
> >> I would be interested in your opinion and if these latest developments
> >> would solve your use-cases.
> >>
> >>> It's definitely nice being able to configure things like this at
> runtime
> >>> for e.g. differentiating between test and production! Would
> functionality
> >>> like this be useful/welcome in docker-galaxy-stable?
> >>
> >> I'm all for it, if this is what is needed by the community.
> >> I would just like to keep the initial startup magic by default as
> >> minimal as possible.
> >>
> >> Thanks a lot for your comments,
> >> Bjoern
> >>
> >>
> >>> Cheers,
> >>>
> >>> Brian
> >>>
> >>
> >

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi Brian, Oksana,

let's move this discussion to github and try to get something done for
the next+1 release.

15.07/15.08 or what ever it will be is really close so I would like to
not rush here. But as soon as we have release this version. We can try
to get something in :dev that addresses these needs.

What do you think? Can you create an Issue for it?
Thanks!
Bjoern

Am 05.08.2015 um 15:39 schrieb Oksana Korol:
> HI Brian,
> 
> Thanks for sharing your code. We have just discussed your approach as one
> possibility of solving our problem, but delayed it as it involved too much
> hacking :). Now that you've done the hacking, we may revisit it.
> We were also discussing a possibility of creating access groups with Galaxy
> UID and cluster process ID to get around the permissions problem that Iyad
> has described. This would involve extra setup outside Galaxy container, but
> would possibly remove the need to change GID and UID. Will reply when we've
> solidified and tested our approach.
> 
> Bjorn,
> 
> I would second Brian on being able to configure Galaxy for different
> environments. To us that was the primary reason of going the docker way. We
> have considered vagrant, since it seemed like a more mature and more
> flexible technology for running in production, but went with docker,
> because Galaxy community supports it.
> 
> I would personally like to be able to change build-time variables at build,
> since it's more logical, requires less hacks and the build speed is not as
> important to me when building containers for different environments, since
> I'm not going to do that often. I.e.I would still have your lightweight way
> as a default, but allow to change it for those who need it. That said, I
> would welcome any standardized solution, be it run time or build time.
> 
> Cheers,
> Oksana
> 
> On Wed, Aug 5, 2015 at 4:30 AM, Björn Grüning 
> wrote:
> 
>> Hi Brian,
>>
>> thanks for you mail! Very appreciated, please see my comments inline.
>>
 We only have one cluster, so we will have both test and production
 environments running on it. I understand that at a minimum we'd need
 different Galaxy home dir and UID/GID.
>>>
>>>
>>> For our local deployment of Galaxy with Docker, which was inspired by a
>>> very early version of Björn's docker-galaxy-stable, we use an entrypoint
>>> script to remap the Galaxy user's UID and GID inside the container from
>>> environment variables GALAXY_UID and GALAXY_GID passed to `docker run`:
>>
>> Is this really needed. We could do this in the main image as well, but
>> currently I haven't seen a compelling usecase for this. The downside is
>> that we need to do a lot of `chown/chmod` magic during container startup.
>> But I'm totally willing to do so, if this is was the community is needed.
>>
>> Please also see this latest PR:
>> https://github.com/bgruening/docker-galaxy-stable/pull/71
>>
>>>
>> https://github.com/fredhutchio/docker-galaxy/blob/aa07c5f684d55499a87e77c62aceda707b63077f/docker-entrypoint.sh#L30
>>>
>>> Just below that, starting at line 42, is an admittedly hacky way of
>>> relocating the Galaxy directory hierarchy (exported data and/or code)
>> from
>>> its container default location /galaxy to another location specified by
>> the
>>> GALAXY_ROOT environment variable. It essentially turns the container into
>>> an installer when `--reroot` or `--upgrade` is passed to the entrypoint
>>> script. `--reroot` copies the container's entire /galaxy hierarchy to
>>> $GALAXY_ROOT for initializing an instance from the container. `--upgrade`
>>> copies just the container's Galaxy distribution in /galaxy/stable to
>>> $GALAXY_ROOT/stable (aka $GALAXY_HOME) for upgrading an existing instance
>>> from a more recent image.
>>
>> Interesting!
>> At least for easier upgrading we took a slightly different approach,
>> described here:
>>
>> https://github.com/bgruening/docker-galaxy-stable/tree/dev#upgrading-images
>>
>> Also we store nowadays many original config files in /etc/galaxy/ to
>> make the upgrading smoother.
>> I would be interested in your opinion and if these latest developments
>> would solve your use-cases.
>>
>>> It's definitely nice being able to configure things like this at runtime
>>> for e.g. differentiating between test and production! Would functionality
>>> like this be useful/welcome in docker-galaxy-stable?
>>
>> I'm all for it, if this is what is needed by the community.
>> I would just like to keep the initial startup magic by default as
>> minimal as possible.
>>
>> Thanks a lot for your comments,
>> Bjoern
>>
>>
>>> Cheers,
>>>
>>> Brian
>>>
>>
> 
> 
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Oksana Korol]

HI Brian,

Thanks for sharing your code. We have just discussed your approach as one
possibility of solving our problem, but delayed it as it involved too much
hacking :). Now that you've done the hacking, we may revisit it.
We were also discussing a possibility of creating access groups with Galaxy
UID and cluster process ID to get around the permissions problem that Iyad
has described. This would involve extra setup outside Galaxy container, but
would possibly remove the need to change GID and UID. Will reply when we've
solidified and tested our approach.

Bjorn,

I would second Brian on being able to configure Galaxy for different
environments. To us that was the primary reason of going the docker way. We
have considered vagrant, since it seemed like a more mature and more
flexible technology for running in production, but went with docker,
because Galaxy community supports it.

I would personally like to be able to change build-time variables at build,
since it's more logical, requires less hacks and the build speed is not as
important to me when building containers for different environments, since
I'm not going to do that often. I.e.I would still have your lightweight way
as a default, but allow to change it for those who need it. That said, I
would welcome any standardized solution, be it run time or build time.

Cheers,
Oksana

On Wed, Aug 5, 2015 at 4:30 AM, Björn Grüning 
wrote:

> Hi Brian,
>
> thanks for you mail! Very appreciated, please see my comments inline.
>
> >> We only have one cluster, so we will have both test and production
> >> environments running on it. I understand that at a minimum we'd need
> >> different Galaxy home dir and UID/GID.
> >
> >
> > For our local deployment of Galaxy with Docker, which was inspired by a
> > very early version of Björn's docker-galaxy-stable, we use an entrypoint
> > script to remap the Galaxy user's UID and GID inside the container from
> > environment variables GALAXY_UID and GALAXY_GID passed to `docker run`:
>
> Is this really needed. We could do this in the main image as well, but
> currently I haven't seen a compelling usecase for this. The downside is
> that we need to do a lot of `chown/chmod` magic during container startup.
> But I'm totally willing to do so, if this is was the community is needed.
>
> Please also see this latest PR:
> https://github.com/bgruening/docker-galaxy-stable/pull/71
>
> >
> https://github.com/fredhutchio/docker-galaxy/blob/aa07c5f684d55499a87e77c62aceda707b63077f/docker-entrypoint.sh#L30
> >
> > Just below that, starting at line 42, is an admittedly hacky way of
> > relocating the Galaxy directory hierarchy (exported data and/or code)
> from
> > its container default location /galaxy to another location specified by
> the
> > GALAXY_ROOT environment variable. It essentially turns the container into
> > an installer when `--reroot` or `--upgrade` is passed to the entrypoint
> > script. `--reroot` copies the container's entire /galaxy hierarchy to
> > $GALAXY_ROOT for initializing an instance from the container. `--upgrade`
> > copies just the container's Galaxy distribution in /galaxy/stable to
> > $GALAXY_ROOT/stable (aka $GALAXY_HOME) for upgrading an existing instance
> > from a more recent image.
>
> Interesting!
> At least for easier upgrading we took a slightly different approach,
> described here:
>
> https://github.com/bgruening/docker-galaxy-stable/tree/dev#upgrading-images
>
> Also we store nowadays many original config files in /etc/galaxy/ to
> make the upgrading smoother.
> I would be interested in your opinion and if these latest developments
> would solve your use-cases.
>
> > It's definitely nice being able to configure things like this at runtime
> > for e.g. differentiating between test and production! Would functionality
> > like this be useful/welcome in docker-galaxy-stable?
>
> I'm all for it, if this is what is needed by the community.
> I would just like to keep the initial startup magic by default as
> minimal as possible.
>
> Thanks a lot for your comments,
> Bjoern
>
>
> > Cheers,
> >
> > Brian
> >
>



-- 
::
Oksana Korol
dataquarry.blogspot.ca
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi Brian,

thanks for you mail! Very appreciated, please see my comments inline.

>> We only have one cluster, so we will have both test and production
>> environments running on it. I understand that at a minimum we'd need
>> different Galaxy home dir and UID/GID.
> 
> 
> For our local deployment of Galaxy with Docker, which was inspired by a
> very early version of Björn's docker-galaxy-stable, we use an entrypoint
> script to remap the Galaxy user's UID and GID inside the container from
> environment variables GALAXY_UID and GALAXY_GID passed to `docker run`:

Is this really needed. We could do this in the main image as well, but
currently I haven't seen a compelling usecase for this. The downside is
that we need to do a lot of `chown/chmod` magic during container startup.
But I'm totally willing to do so, if this is was the community is needed.

Please also see this latest PR:
https://github.com/bgruening/docker-galaxy-stable/pull/71

> https://github.com/fredhutchio/docker-galaxy/blob/aa07c5f684d55499a87e77c62aceda707b63077f/docker-entrypoint.sh#L30
> 
> Just below that, starting at line 42, is an admittedly hacky way of
> relocating the Galaxy directory hierarchy (exported data and/or code) from
> its container default location /galaxy to another location specified by the
> GALAXY_ROOT environment variable. It essentially turns the container into
> an installer when `--reroot` or `--upgrade` is passed to the entrypoint
> script. `--reroot` copies the container's entire /galaxy hierarchy to
> $GALAXY_ROOT for initializing an instance from the container. `--upgrade`
> copies just the container's Galaxy distribution in /galaxy/stable to
> $GALAXY_ROOT/stable (aka $GALAXY_HOME) for upgrading an existing instance
> from a more recent image.

Interesting!
At least for easier upgrading we took a slightly different approach,
described here:

https://github.com/bgruening/docker-galaxy-stable/tree/dev#upgrading-images

Also we store nowadays many original config files in /etc/galaxy/ to
make the upgrading smoother.
I would be interested in your opinion and if these latest developments
would solve your use-cases.

> It's definitely nice being able to configure things like this at runtime
> for e.g. differentiating between test and production! Would functionality
> like this be useful/welcome in docker-galaxy-stable?

I'm all for it, if this is what is needed by the community.
I would just like to keep the initial startup magic by default as
minimal as possible.

Thanks a lot for your comments,
Bjoern


> Cheers,
> 
> Brian
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Brian Claywell]

On Sun, Aug 2, 2015 at 4:59 AM, Oksana  wrote:

> We only have one cluster, so we will have both test and production
> environments running on it. I understand that at a minimum we'd need
> different Galaxy home dir and UID/GID.


For our local deployment of Galaxy with Docker, which was inspired by a
very early version of Björn's docker-galaxy-stable, we use an entrypoint
script to remap the Galaxy user's UID and GID inside the container from
environment variables GALAXY_UID and GALAXY_GID passed to `docker run`:

https://github.com/fredhutchio/docker-galaxy/blob/aa07c5f684d55499a87e77c62aceda707b63077f/docker-entrypoint.sh#L30

Just below that, starting at line 42, is an admittedly hacky way of
relocating the Galaxy directory hierarchy (exported data and/or code) from
its container default location /galaxy to another location specified by the
GALAXY_ROOT environment variable. It essentially turns the container into
an installer when `--reroot` or `--upgrade` is passed to the entrypoint
script. `--reroot` copies the container's entire /galaxy hierarchy to
$GALAXY_ROOT for initializing an instance from the container. `--upgrade`
copies just the container's Galaxy distribution in /galaxy/stable to
$GALAXY_ROOT/stable (aka $GALAXY_HOME) for upgrading an existing instance
from a more recent image.

It's definitely nice being able to configure things like this at runtime
for e.g. differentiating between test and production! Would functionality
like this be useful/welcome in docker-galaxy-stable?


Cheers,

Brian

-- 
Brian Claywell | programmer/analyst
Matsen Group   | http://matsen.fredhutch.org 
Fred Hutchinson Cancer Research Center
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi Iyad,

does this part of our documentation help you?

https://github.com/bgruening/docker-galaxy-stable/tree/dev#using-an-external-slurm-cluster

I now at least of one user that is using the Docker Image in a SLURM
cluster environment. I don't see a reason why this should not work on
SGE or condor as well.

Ciao,
Bjoern

> Hi Everyone,
> 
> As a supplement to Oksana's original inquiry, we, at AAFC, couple
> Galaxy to SGE  and possibly a different grid engine in the future for
> job scheduling.  The old-fashioned way of doing that is to unify
> storage path and UID/GID between the application server and cluster.
> The galaxy wiki page refers to this as a "shared file system".  I
> suppose we can configure SGE to submit jobs as a specific user but
> the resultant files would need to be chowned to the user within the
> cluster and then back to the one inside the container.  Is this
> supported or has been tested?
> 
> Regards,
> 
> Iyad Kandalaft
> 
> Acting Chief Bioinformatician in Biodiversity, STB Agriculture and
> Agri-Food Canada / Government of Canada iyad.kandal...@agr.gc.ca /
> Tel: 613-759-1228 / TTY: 613-773-2600
> 
> Bioinformaticien chef de la  biodiversite interim, Direction générale
> des Science et de la technologie Agriculture et Agroalimentaire
> Canada / Gouvernement du Canada iyad.kandal...@agr.gc.ca / Tel:
> 613-759-1228 / TTY: 613-773-2600
> 
> 
> 
> 
> -Original Message- From: galaxy-dev
> [mailto:galaxy-dev-boun...@lists.galaxyproject.org] On Behalf Of
> Björn Grüning Sent: August-04-15 4:42 AM To: Tiago Rodrigues Antao;
> Björn Grüning Cc: galaxy-...@lists.bx.psu.edu; Martin Page (TSL);
> Oksana; Jeroen Galle Subject: Re: [galaxy-dev] Externalizing Galaxy
> config in docker-galaxy-stable
> 
> 
> 
> Am 04.08.2015 um 00:00 schrieb Tiago Rodrigues Antao:
>> On Mon, 03 Aug 2015 17:30:05 +0200 Björn Grüning
>>  wrote:
>> 
>>> I would recommend you Galaxy Flavours. We have a few, for
>>> example RNA-Seq, ChIP-seq etc ... join us! :)
>> 
>> That was my first port of call, but they lacked quite a few tools.
>> For example the RNA-Seq lacked tophat, cufflinks and cummrbund. I
>> wanted to reuse as much as possible, but that was what made me
>> stop.
> 
> Create a PR! 
> https://github.com/bgruening/docker-recipes/blob/master/galaxy-rna-seq/Dockerfile
>
> 
>> I am currently using the stable one (docker-galaxy-stable) and
>> advancing from there. But I would prefer to use a "stock" one.
> 
> If you feel I can move this into it's own repository and out of 
> docker-recipes. PR welcome! HiSat should be also included I guess.
> 
> I hope to make this flavours community driven, so I really
> appreciate contributions to it.
> 
> Ciao, Bjoern
> 
>> Tiago
>> 
> ___ Please
> keep all replies on the list by using "reply all" in your mail
> client.  To manage your subscriptions to this and other Galaxy lists,
> please use the interface at: https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at: 
> http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Kandalaft, Iyad]

Hi Everyone,

As a supplement to Oksana's original inquiry, we, at AAFC, couple Galaxy to SGE 
 and possibly a different grid engine in the future for job scheduling.  The 
old-fashioned way of doing that is to unify storage path and UID/GID between 
the application server and cluster.  The galaxy wiki page refers to this as a 
"shared file system".  I suppose we can configure SGE to submit jobs as a 
specific user but the resultant files would need to be chowned to the user 
within the cluster and then back to the one inside the container.  Is this 
supported or has been tested?

Regards,

Iyad Kandalaft

Acting Chief Bioinformatician in Biodiversity, STB
Agriculture and Agri-Food Canada / Government of Canada
iyad.kandal...@agr.gc.ca / Tel: 613-759-1228 / TTY: 613-773-2600

Bioinformaticien chef de la  biodiversite interim, Direction générale des 
Science et de la technologie
Agriculture et Agroalimentaire Canada / Gouvernement du Canada
iyad.kandal...@agr.gc.ca / Tel: 613-759-1228 / TTY: 613-773-2600
 



-Original Message-
From: galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] On Behalf 
Of Björn Grüning
Sent: August-04-15 4:42 AM
To: Tiago Rodrigues Antao; Björn Grüning
Cc: galaxy-...@lists.bx.psu.edu; Martin Page (TSL); Oksana; Jeroen Galle
Subject: Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable



Am 04.08.2015 um 00:00 schrieb Tiago Rodrigues Antao:
> On Mon, 03 Aug 2015 17:30:05 +0200
> Björn Grüning  wrote:
> 
>> I would recommend you Galaxy Flavours. We have a few, for example
>> RNA-Seq, ChIP-seq etc ... join us! :)
> 
> That was my first port of call, but they lacked quite a few tools. For
> example the RNA-Seq lacked tophat, cufflinks and cummrbund. I wanted to
> reuse as much as possible, but that was what made me stop.

Create a PR!
https://github.com/bgruening/docker-recipes/blob/master/galaxy-rna-seq/Dockerfile

> I am currently using the stable one (docker-galaxy-stable) and advancing
> from there. But I would prefer to use a "stock" one.

If you feel I can move this into it's own repository and out of
docker-recipes. PR welcome! HiSat should be also included I guess.

I hope to make this flavours community driven, so I really appreciate
contributions to it.

Ciao,
Bjoern

> Tiago
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Am 04.08.2015 um 00:00 schrieb Tiago Rodrigues Antao:
> On Mon, 03 Aug 2015 17:30:05 +0200
> Björn Grüning  wrote:
> 
>> I would recommend you Galaxy Flavours. We have a few, for example
>> RNA-Seq, ChIP-seq etc ... join us! :)
> 
> That was my first port of call, but they lacked quite a few tools. For
> example the RNA-Seq lacked tophat, cufflinks and cummrbund. I wanted to
> reuse as much as possible, but that was what made me stop.

Create a PR!
https://github.com/bgruening/docker-recipes/blob/master/galaxy-rna-seq/Dockerfile

> I am currently using the stable one (docker-galaxy-stable) and advancing
> from there. But I would prefer to use a "stock" one.

If you feel I can move this into it's own repository and out of
docker-recipes. PR welcome! HiSat should be also included I guess.

I hope to make this flavours community driven, so I really appreciate
contributions to it.

Ciao,
Bjoern

> Tiago
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

On Mon, 03 Aug 2015 17:30:05 +0200
Björn Grüning  wrote:

> I would recommend you Galaxy Flavours. We have a few, for example
> RNA-Seq, ChIP-seq etc ... join us! :)

That was my first port of call, but they lacked quite a few tools. For
example the RNA-Seq lacked tophat, cufflinks and cummrbund. I wanted to
reuse as much as possible, but that was what made me stop.

I am currently using the stable one (docker-galaxy-stable) and advancing
from there. But I would prefer to use a "stock" one.

Tiago
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi Tiago,

Am 03.08.2015 um 15:44 schrieb Tiago Rodrigues Antao:
> Hi,
> 
> I am also thinking in doing something similar.
> 
> My idea would be to deploy a galaxy docker on demand per project. The
> docker would have a galaxy installation that would be configured to
> join a cluster. Most of the computation would actually occur on the
> cluster.

!!!
I know of a computer centers that are developing an interface where the
user can choose a Docker container, based on their needs.
So this is definitely possible

> Each project would thus have its isolated galaxy with separate users. I
> am extending your docker file with all the tools from the toolshed (I
> know, it will become a large one-size-fits-all installation).

I would recommend you Galaxy Flavours. We have a few, for example
RNA-Seq, ChIP-seq etc ... join us! :)

> If there is any opinion on this approach, it would be appreciated.

I don't think on container that fits all is good for usability, let's
create small special ones.

> I am actually thinking in developing a very small GUI to configure the
> Dockerfile (say with users and cluster configuration) before build/run.

Uh nice!
Martin Page and Jeroen Galle are currently working on a Galaxy flavour
generator:

http://wookoouk.github.io/galaxyFlavorGenerator/

Maybe we could exchange ideas here?

Thanks for you mail,
Bjoern

> Tiago
> 
> 
> On Sun, 02 Aug 2015 10:23:20 +0200
> Björn Grüning  wrote:
> 
>> Hi,
>>
>> I will think about this, but this comes with the price of doing much
>> of the internal stuff during startup, which also have it's downsides.
>>
>> Can you tell me what your admin wants to do, maybe we find a solution
>> without changing the build-time variables?
>> For example we can talk to schedulers without changing UID/GID ...
>>
>> Thanks for using Galaxy!
>> Bjoern
>>
>> Am 02.08.2015 um 05:19 schrieb Oksana:
>>> Hi Bjorn,
>>>
>>> Our team is trying to streamline our Galaxy development process, by
>>> using your docker container for development, test and production
>>> environments. To do that, our sys admin needs to be able to change a
>>> few of Galaxy build time variables, like the ones that I've
>>> mentioned and probably more, since our production is running on a
>>> cluster and its setup will be quite different from the local dev.
>>> We wouldn't want to modify the docker image from environment to
>>> environment, so a config file, which would allow to set those build
>>> variables before a container is built, would be really useful. Are
>>> you planning to extend your development to accommodate this need in
>>> the near future? Or if there is already a way to do this that I
>>> don't see, please let me know.
>>>
>>> Cheers, Oksana
>>>
>>>
>>>
 On Aug 1, 2015, at 1:10 PM, Björn Grüning
  wrote:

 Hi Oksana,

 both variables are used during container build to create users and
 home directories. This does not happen during container startup.
 They are buildtime variables.

 All Galaxy ENV vars are runtime variables and do have an impact
 during startup.

 Can you tell us more about what you want to do? Thanks, Bjoern

> Hi,
>
> My question is to the docker-galaxy-stable community. I would
> like to use or extend this image with some Galaxy settings
> externalized. For instance, I would like to define my own
> GALAXY_UID and GALAXY_HOME environment variables. I have tried
> the -e setting when I build and run the container, but that
> doesn't work:
>
>> docker run -d -p 8080:80 -p 8021:21 -e "GALAXY_UID=1777" -e
> "GALAXY_HOME=/home/galaxy/env_test" --name galaxy-env-test 
> bgruening/galaxy-stable ...
>> docker exec -ti  galaxy-env-test bash # getent passwd galaxy
> galaxy:x:1450:1450:Galaxy user:/home/galaxy:
>
> As you can see from above, GALAXY_UID is 1450, as hardcoded in
> the Dockerfile, and not 1777, as I've specified. Same goes for
> the home directory.
>
> Is there any other way that I can set those variables? If not,
> what would be the best way to proceed, since, ideally, I would
> like to extend the galaxy-stable Docker image, and not change the
> existing one. Currently, I don't see any other way but to fork 
> https://github.com/bgruening/docker-galaxy-stable and change the
> Dockerfile to externalize those (and other) variables. I hope I
> can get better suggestions than this.
>
> Cheers, Oksana
>
>
>
> ___ 
> Please keep all replies on the list by using "reply all" in your
> mail client.  To manage your subscriptions to this and other
> Galaxy lists, please use the interface at: 
> https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at: 
> http://galaxyproject.org/search/mailinglists/
>
>> ___
>> Plea

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

Hi,

I am also thinking in doing something similar.

My idea would be to deploy a galaxy docker on demand per project. The
docker would have a galaxy installation that would be configured to
join a cluster. Most of the computation would actually occur on the
cluster.

Each project would thus have its isolated galaxy with separate users. I
am extending your docker file with all the tools from the toolshed (I
know, it will become a large one-size-fits-all installation).

If there is any opinion on this approach, it would be appreciated.

I am actually thinking in developing a very small GUI to configure the
Dockerfile (say with users and cluster configuration) before build/run.

Tiago


On Sun, 02 Aug 2015 10:23:20 +0200
Björn Grüning  wrote:

> Hi,
> 
> I will think about this, but this comes with the price of doing much
> of the internal stuff during startup, which also have it's downsides.
> 
> Can you tell me what your admin wants to do, maybe we find a solution
> without changing the build-time variables?
> For example we can talk to schedulers without changing UID/GID ...
> 
> Thanks for using Galaxy!
> Bjoern
> 
> Am 02.08.2015 um 05:19 schrieb Oksana:
> > Hi Bjorn,
> > 
> > Our team is trying to streamline our Galaxy development process, by
> > using your docker container for development, test and production
> > environments. To do that, our sys admin needs to be able to change a
> > few of Galaxy build time variables, like the ones that I've
> > mentioned and probably more, since our production is running on a
> > cluster and its setup will be quite different from the local dev.
> > We wouldn't want to modify the docker image from environment to
> > environment, so a config file, which would allow to set those build
> > variables before a container is built, would be really useful. Are
> > you planning to extend your development to accommodate this need in
> > the near future? Or if there is already a way to do this that I
> > don't see, please let me know.
> > 
> > Cheers, Oksana
> > 
> > 
> > 
> >> On Aug 1, 2015, at 1:10 PM, Björn Grüning
> >>  wrote:
> >> 
> >> Hi Oksana,
> >> 
> >> both variables are used during container build to create users and
> >> home directories. This does not happen during container startup.
> >> They are buildtime variables.
> >> 
> >> All Galaxy ENV vars are runtime variables and do have an impact
> >> during startup.
> >> 
> >> Can you tell us more about what you want to do? Thanks, Bjoern
> >> 
> >>> Hi,
> >>> 
> >>> My question is to the docker-galaxy-stable community. I would
> >>> like to use or extend this image with some Galaxy settings
> >>> externalized. For instance, I would like to define my own
> >>> GALAXY_UID and GALAXY_HOME environment variables. I have tried
> >>> the -e setting when I build and run the container, but that
> >>> doesn't work:
> >>> 
>  docker run -d -p 8080:80 -p 8021:21 -e "GALAXY_UID=1777" -e
> >>> "GALAXY_HOME=/home/galaxy/env_test" --name galaxy-env-test 
> >>> bgruening/galaxy-stable ...
>  docker exec -ti  galaxy-env-test bash # getent passwd galaxy
> >>> galaxy:x:1450:1450:Galaxy user:/home/galaxy:
> >>> 
> >>> As you can see from above, GALAXY_UID is 1450, as hardcoded in
> >>> the Dockerfile, and not 1777, as I've specified. Same goes for
> >>> the home directory.
> >>> 
> >>> Is there any other way that I can set those variables? If not,
> >>> what would be the best way to proceed, since, ideally, I would
> >>> like to extend the galaxy-stable Docker image, and not change the
> >>> existing one. Currently, I don't see any other way but to fork 
> >>> https://github.com/bgruening/docker-galaxy-stable and change the
> >>> Dockerfile to externalize those (and other) variables. I hope I
> >>> can get better suggestions than this.
> >>> 
> >>> Cheers, Oksana
> >>> 
> >>> 
> >>> 
> >>> ___ 
> >>> Please keep all replies on the list by using "reply all" in your
> >>> mail client.  To manage your subscriptions to this and other
> >>> Galaxy lists, please use the interface at: 
> >>> https://lists.galaxyproject.org/
> >>> 
> >>> To search Galaxy mailing lists use the unified search at: 
> >>> http://galaxyproject.org/search/mailinglists/
> >>> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/sear

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Oksana]

We only have one cluster, so we will have both test and production environments 
running on it. I understand that at a minimum we'd need different Galaxy home 
dir and UID/GID. This is a proof of concept for us still, so I can't tell you 
what else we might need to change. I can get hold of our admin on Tuesday and 
he can clarify if there any other build time variables he needs to change. I'm 
running with the local dev version and so far it's been great. I just need a 
local toolshed situation figured out, to be able to install my tools. 

Thanks for maintaining Galaxy! Will try to contribute once I get more 
up-to-speed.

Cheers,
Oksana


> On Aug 2, 2015, at 4:23 AM, Björn Grüning  wrote:
> 
> Hi,
> 
> I will think about this, but this comes with the price of doing much of
> the internal stuff during startup, which also have it's downsides.
> 
> Can you tell me what your admin wants to do, maybe we find a solution
> without changing the build-time variables?
> For example we can talk to schedulers without changing UID/GID ...
> 
> Thanks for using Galaxy!
> Bjoern
> 
>> Am 02.08.2015 um 05:19 schrieb Oksana:
>> Hi Bjorn,
>> 
>> Our team is trying to streamline our Galaxy development process, by
>> using your docker container for development, test and production
>> environments. To do that, our sys admin needs to be able to change a
>> few of Galaxy build time variables, like the ones that I've mentioned
>> and probably more, since our production is running on a cluster and
>> its setup will be quite different from the local dev. We wouldn't
>> want to modify the docker image from environment to environment, so a
>> config file, which would allow to set those build variables before a
>> container is built, would be really useful. Are you planning to
>> extend your development to accommodate this need in the near future?
>> Or if there is already a way to do this that I don't see, please let
>> me know.
>> 
>> Cheers, Oksana
>> 
>> 
>> 
>>> On Aug 1, 2015, at 1:10 PM, Björn Grüning
>>>  wrote:
>>> 
>>> Hi Oksana,
>>> 
>>> both variables are used during container build to create users and
>>> home directories. This does not happen during container startup.
>>> They are buildtime variables.
>>> 
>>> All Galaxy ENV vars are runtime variables and do have an impact
>>> during startup.
>>> 
>>> Can you tell us more about what you want to do? Thanks, Bjoern
>>> 
 Hi,
 
 My question is to the docker-galaxy-stable community. I would
 like to use or extend this image with some Galaxy settings
 externalized. For instance, I would like to define my own
 GALAXY_UID and GALAXY_HOME environment variables. I have tried
 the -e setting when I build and run the container, but that
 doesn't work:
 
> docker run -d -p 8080:80 -p 8021:21 -e "GALAXY_UID=1777" -e
 "GALAXY_HOME=/home/galaxy/env_test" --name galaxy-env-test 
 bgruening/galaxy-stable ...
> docker exec -ti  galaxy-env-test bash # getent passwd galaxy
 galaxy:x:1450:1450:Galaxy user:/home/galaxy:
 
 As you can see from above, GALAXY_UID is 1450, as hardcoded in
 the Dockerfile, and not 1777, as I've specified. Same goes for
 the home directory.
 
 Is there any other way that I can set those variables? If not,
 what would be the best way to proceed, since, ideally, I would
 like to extend the galaxy-stable Docker image, and not change the
 existing one. Currently, I don't see any other way but to fork 
 https://github.com/bgruening/docker-galaxy-stable and change the
 Dockerfile to externalize those (and other) variables. I hope I
 can get better suggestions than this.
 
 Cheers, Oksana
 
 
 
 ___ 
 Please keep all replies on the list by using "reply all" in your
 mail client.  To manage your subscriptions to this and other
 Galaxy lists, please use the interface at: 
 https://lists.galaxyproject.org/
 
 To search Galaxy mailing lists use the unified search at: 
 http://galaxyproject.org/search/mailinglists/
 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi,

I will think about this, but this comes with the price of doing much of
the internal stuff during startup, which also have it's downsides.

Can you tell me what your admin wants to do, maybe we find a solution
without changing the build-time variables?
For example we can talk to schedulers without changing UID/GID ...

Thanks for using Galaxy!
Bjoern

Am 02.08.2015 um 05:19 schrieb Oksana:
> Hi Bjorn,
> 
> Our team is trying to streamline our Galaxy development process, by
> using your docker container for development, test and production
> environments. To do that, our sys admin needs to be able to change a
> few of Galaxy build time variables, like the ones that I've mentioned
> and probably more, since our production is running on a cluster and
> its setup will be quite different from the local dev. We wouldn't
> want to modify the docker image from environment to environment, so a
> config file, which would allow to set those build variables before a
> container is built, would be really useful. Are you planning to
> extend your development to accommodate this need in the near future?
> Or if there is already a way to do this that I don't see, please let
> me know.
> 
> Cheers, Oksana
> 
> 
> 
>> On Aug 1, 2015, at 1:10 PM, Björn Grüning
>>  wrote:
>> 
>> Hi Oksana,
>> 
>> both variables are used during container build to create users and
>> home directories. This does not happen during container startup.
>> They are buildtime variables.
>> 
>> All Galaxy ENV vars are runtime variables and do have an impact
>> during startup.
>> 
>> Can you tell us more about what you want to do? Thanks, Bjoern
>> 
>>> Hi,
>>> 
>>> My question is to the docker-galaxy-stable community. I would
>>> like to use or extend this image with some Galaxy settings
>>> externalized. For instance, I would like to define my own
>>> GALAXY_UID and GALAXY_HOME environment variables. I have tried
>>> the -e setting when I build and run the container, but that
>>> doesn't work:
>>> 
 docker run -d -p 8080:80 -p 8021:21 -e "GALAXY_UID=1777" -e
>>> "GALAXY_HOME=/home/galaxy/env_test" --name galaxy-env-test 
>>> bgruening/galaxy-stable ...
 docker exec -ti  galaxy-env-test bash # getent passwd galaxy
>>> galaxy:x:1450:1450:Galaxy user:/home/galaxy:
>>> 
>>> As you can see from above, GALAXY_UID is 1450, as hardcoded in
>>> the Dockerfile, and not 1777, as I've specified. Same goes for
>>> the home directory.
>>> 
>>> Is there any other way that I can set those variables? If not,
>>> what would be the best way to proceed, since, ideally, I would
>>> like to extend the galaxy-stable Docker image, and not change the
>>> existing one. Currently, I don't see any other way but to fork 
>>> https://github.com/bgruening/docker-galaxy-stable and change the
>>> Dockerfile to externalize those (and other) variables. I hope I
>>> can get better suggestions than this.
>>> 
>>> Cheers, Oksana
>>> 
>>> 
>>> 
>>> ___ 
>>> Please keep all replies on the list by using "reply all" in your
>>> mail client.  To manage your subscriptions to this and other
>>> Galaxy lists, please use the interface at: 
>>> https://lists.galaxyproject.org/
>>> 
>>> To search Galaxy mailing lists use the unified search at: 
>>> http://galaxyproject.org/search/mailinglists/
>>> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Oksana]

Hi Bjorn,

Our team is trying to streamline our Galaxy development process, by using your 
docker container for development, test and production environments. To do that, 
our sys admin needs to be able to change a few of Galaxy build time variables, 
like the ones that I've mentioned and probably more, since our production is 
running on a cluster and its setup will be quite different from the local dev. 
We wouldn't want to modify the docker image from environment to environment, so 
a config file, which would allow to set those build variables before a 
container is built, would be really useful. Are you planning to extend your 
development to accommodate this need in the near future? Or if there is already 
a way to do this that I don't see, please let me know.

Cheers,
Oksana



> On Aug 1, 2015, at 1:10 PM, Björn Grüning  wrote:
> 
> Hi Oksana,
> 
> both variables are used during container build to create users and home
> directories. This does not happen during container startup. They are
> buildtime variables.
> 
> All Galaxy ENV vars are runtime variables and do have an impact during
> startup.
> 
> Can you tell us more about what you want to do?
> Thanks,
> Bjoern
> 
>> Hi,
>> 
>> My question is to the docker-galaxy-stable community. I would like to use
>> or extend this image with some Galaxy settings externalized. For instance,
>> I would like to define my own GALAXY_UID and GALAXY_HOME environment
>> variables. I have tried the -e setting when I build and run the container,
>> but that doesn't work:
>> 
>>> docker run -d -p 8080:80 -p 8021:21 -e "GALAXY_UID=1777" -e
>> "GALAXY_HOME=/home/galaxy/env_test" --name galaxy-env-test
>> bgruening/galaxy-stable
>> ...
>>> docker exec -ti  galaxy-env-test bash
>>> # getent passwd galaxy
>> galaxy:x:1450:1450:Galaxy user:/home/galaxy:
>> 
>> As you can see from above, GALAXY_UID is 1450, as hardcoded in the
>> Dockerfile, and not 1777, as I've specified. Same goes for the home
>> directory.
>> 
>> Is there any other way that I can set those variables? If not, what would
>> be the best way to proceed, since, ideally, I would like to extend the
>> galaxy-stable Docker image, and not change the existing one. Currently, I
>> don't see any other way but to fork
>> https://github.com/bgruening/docker-galaxy-stable and change the Dockerfile
>> to externalize those (and other) variables. I hope I can get better
>> suggestions than this.
>> 
>> Cheers,
>> Oksana
>> 
>> 
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  https://lists.galaxyproject.org/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/
>> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Björn Grüning]

Hi Oksana,

both variables are used during container build to create users and home
directories. This does not happen during container startup. They are
buildtime variables.

All Galaxy ENV vars are runtime variables and do have an impact during
startup.

Can you tell us more about what you want to do?
Thanks,
Bjoern

> Hi,
> 
> My question is to the docker-galaxy-stable community. I would like to use
> or extend this image with some Galaxy settings externalized. For instance,
> I would like to define my own GALAXY_UID and GALAXY_HOME environment
> variables. I have tried the -e setting when I build and run the container,
> but that doesn't work:
> 
>> docker run -d -p 8080:80 -p 8021:21 -e "GALAXY_UID=1777" -e
> "GALAXY_HOME=/home/galaxy/env_test" --name galaxy-env-test
> bgruening/galaxy-stable
> ...
>> docker exec -ti  galaxy-env-test bash
>> # getent passwd galaxy
> galaxy:x:1450:1450:Galaxy user:/home/galaxy:
> 
>  As you can see from above, GALAXY_UID is 1450, as hardcoded in the
> Dockerfile, and not 1777, as I've specified. Same goes for the home
> directory.
> 
> Is there any other way that I can set those variables? If not, what would
> be the best way to proceed, since, ideally, I would like to extend the
> galaxy-stable Docker image, and not change the existing one. Currently, I
> don't see any other way but to fork
> https://github.com/bgruening/docker-galaxy-stable and change the Dockerfile
> to externalize those (and other) variables. I hope I can get better
> suggestions than this.
> 
> Cheers,
> Oksana
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Oksana Korol]

Hi,

My question is to the docker-galaxy-stable community. I would like to use
or extend this image with some Galaxy settings externalized. For instance,
I would like to define my own GALAXY_UID and GALAXY_HOME environment
variables. I have tried the -e setting when I build and run the container,
but that doesn't work:

> docker run -d -p 8080:80 -p 8021:21 -e "GALAXY_UID=1777" -e
"GALAXY_HOME=/home/galaxy/env_test" --name galaxy-env-test
bgruening/galaxy-stable
...
> docker exec -ti  galaxy-env-test bash
># getent passwd galaxy
galaxy:x:1450:1450:Galaxy user:/home/galaxy:

 As you can see from above, GALAXY_UID is 1450, as hardcoded in the
Dockerfile, and not 1777, as I've specified. Same goes for the home
directory.

Is there any other way that I can set those variables? If not, what would
be the best way to proceed, since, ideally, I would like to extend the
galaxy-stable Docker image, and not change the existing one. Currently, I
don't see any other way but to fork
https://github.com/bgruening/docker-galaxy-stable and change the Dockerfile
to externalize those (and other) variables. I hope I can get better
suggestions than this.

Cheers,
Oksana
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/