Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

On Mon, 03 Aug 2015 17:30:05 +0200
Björn Grüning bjoern.gruen...@gmail.com wrote:

 I would recommend you Galaxy Flavours. We have a few, for example
 RNA-Seq, ChIP-seq etc ... join us! :)

That was my first port of call, but they lacked quite a few tools. For
example the RNA-Seq lacked tophat, cufflinks and cummrbund. I wanted to
reuse as much as possible, but that was what made me stop.

I am currently using the stable one (docker-galaxy-stable) and advancing
from there. But I would prefer to use a stock one.

Tiago
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

On Tue, 4 Aug 2015 17:26:00 +
Kandalaft, Iyad iyad.kandal...@agr.gc.ca wrote:

 As a supplement to Oksana's original inquiry, we, at AAFC, couple
 Galaxy to SGE  and possibly a different grid engine in the future for
 job scheduling.  The old-fashioned way of doing that is to unify
 storage path and UID/GID between the application server and cluster.


I am starting to warm of the idea of making a small web interface that
asks for a few configuration options:
1. type of grid engine
2. desired docker/galaxy flavour (RNA, exome, WGS, ...)
3. ... (e.g. intial admin user and password)

and would pull Björn's relevant Dockerfile, patch it and generate a new
one, tailored to the need of the user.

I think it is impossible to have a set of dockerfiles for all the
desired configurations (think analysis flavours times number of cluster
types - for example).

I will need to implement something like this very soon, and I would not
mind to do it in a way that the community could reuse...

So comments would be appreciated...
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Externalizing Galaxy config in docker-galaxy-stable

[Tiago Rodrigues Antao]

Hi,

I am also thinking in doing something similar.

My idea would be to deploy a galaxy docker on demand per project. The
docker would have a galaxy installation that would be configured to
join a cluster. Most of the computation would actually occur on the
cluster.

Each project would thus have its isolated galaxy with separate users. I
am extending your docker file with all the tools from the toolshed (I
know, it will become a large one-size-fits-all installation).

If there is any opinion on this approach, it would be appreciated.

I am actually thinking in developing a very small GUI to configure the
Dockerfile (say with users and cluster configuration) before build/run.

Tiago


On Sun, 02 Aug 2015 10:23:20 +0200
Björn Grüning bjoern.gruen...@gmail.com wrote:

 Hi,
 
 I will think about this, but this comes with the price of doing much
 of the internal stuff during startup, which also have it's downsides.
 
 Can you tell me what your admin wants to do, maybe we find a solution
 without changing the build-time variables?
 For example we can talk to schedulers without changing UID/GID ...
 
 Thanks for using Galaxy!
 Bjoern
 
 Am 02.08.2015 um 05:19 schrieb Oksana:
  Hi Bjorn,
  
  Our team is trying to streamline our Galaxy development process, by
  using your docker container for development, test and production
  environments. To do that, our sys admin needs to be able to change a
  few of Galaxy build time variables, like the ones that I've
  mentioned and probably more, since our production is running on a
  cluster and its setup will be quite different from the local dev.
  We wouldn't want to modify the docker image from environment to
  environment, so a config file, which would allow to set those build
  variables before a container is built, would be really useful. Are
  you planning to extend your development to accommodate this need in
  the near future? Or if there is already a way to do this that I
  don't see, please let me know.
  
  Cheers, Oksana
  
  
  
  On Aug 1, 2015, at 1:10 PM, Björn Grüning
  bjoern.gruen...@gmail.com wrote:
  
  Hi Oksana,
  
  both variables are used during container build to create users and
  home directories. This does not happen during container startup.
  They are buildtime variables.
  
  All Galaxy ENV vars are runtime variables and do have an impact
  during startup.
  
  Can you tell us more about what you want to do? Thanks, Bjoern
  
  Hi,
  
  My question is to the docker-galaxy-stable community. I would
  like to use or extend this image with some Galaxy settings
  externalized. For instance, I would like to define my own
  GALAXY_UID and GALAXY_HOME environment variables. I have tried
  the -e setting when I build and run the container, but that
  doesn't work:
  
  docker run -d -p 8080:80 -p 8021:21 -e GALAXY_UID=1777 -e
  GALAXY_HOME=/home/galaxy/env_test --name galaxy-env-test 
  bgruening/galaxy-stable ...
  docker exec -ti  galaxy-env-test bash # getent passwd galaxy
  galaxy:x:1450:1450:Galaxy user:/home/galaxy:
  
  As you can see from above, GALAXY_UID is 1450, as hardcoded in
  the Dockerfile, and not 1777, as I've specified. Same goes for
  the home directory.
  
  Is there any other way that I can set those variables? If not,
  what would be the best way to proceed, since, ideally, I would
  like to extend the galaxy-stable Docker image, and not change the
  existing one. Currently, I don't see any other way but to fork 
  https://github.com/bgruening/docker-galaxy-stable and change the
  Dockerfile to externalize those (and other) variables. I hope I
  can get better suggestions than this.
  
  Cheers, Oksana
  
  
  
  ___ 
  Please keep all replies on the list by using reply all in your
  mail client.  To manage your subscriptions to this and other
  Galaxy lists, please use the interface at: 
  https://lists.galaxyproject.org/
  
  To search Galaxy mailing lists use the unified search at: 
  http://galaxyproject.org/search/mailinglists/
  
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Developing package wrappers

[Tiago Rodrigues Antao]

Hi,

I am afraid that I might be overlooking something obvious but...

I am trying to add a few tools to the RAD-Seq portfolio of Galaxy. For
that effect I am designing a few package wrappers. My problem is to
understand the best practice to develop wrappers for dependencies of
external tools.

For normal tools, that is quite easy: one uses the galaxy/tool/XXX
to do all development and testing.

But for dependency wrappers, the only solution that I have found is to
upload the wrapper to a toolshed and installing it via the admin
interface... This is cumbersome. Is there any way to test/develop a
package wrapper so that I can test tool_dependencies.xml without
uploading it to a toolshed and installing it from there?

Thanks,
Tiago
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Developing package wrappers

[Tiago Rodrigues Antao]

On Tue, 20 Oct 2015 14:48:08 +0200
Bjoern Gruening  wrote:

> Hi Tiago,
> 
> for simple tool_dependencies you can use the new planemo depbash
> command from Peter:
> https://github.com/galaxyproject/planemo/pull/310

Gosh, this is exactly what the doctor ordered. Thanks (to Peter also).

 
> What kind of dependencies do you have? Python/Perl?
> Consider contributing to tools-iuc 
> (https://github.com/galaxyproject/tools-iuc). We can probably help
> you and guide you your way.
> 
> Thanks for working on RAD-seq!


I am actually updating the raxml tool for now.
But in my new position I am heavily involved with RAD-seq data and I
am working on some new galaxy tools to help with more recent analytical
methods (e.g. integrating cap3 with stacks). If there is any reported
need on the RAD-seq front, email me and I might be able to help...

Tiago
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] GATK issues

[Tiago Rodrigues Antao]

Hi,

I have a couple of GATK issues

1. First is there a data manager for it? I have tried Generate
GATK-sorted Picard indexes (data_manager_gatk_picard_index_builder) -
builder and got no entries on the cached picard genomes.

2. With problem 1, we are using genomes from the history. It so happens
that some XML files have the filtering of forcing the dbkey of
the from-history genome to be the same as the BAM files (reduced below
for easy reading):


  
  

  

  


For example, here:
https://github.com/galaxyproject/tools-devteam/blob/master/tool_collections/gatk/realigner_target_creator/realigner_target_creator.xml#L69

Now, this does not seem to work. I.e. when I make the dbkey of the BAM
file and the FASTA file the same, the ref_file entry is still empty.

Also, some xml files have it, others do not.

As a temporary measure, I removed the options/filter in my local
installation, but might this be a bug? Or is it a bug in my head?

Thanks,
Tiago
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/