On 19 August 2014 14:07, Björn Grüning wrote:
> Hi Marius,
>
> thanks for shifting the discussion over to the mailinglist.
>
> Am 19.08.2014 um 13:13 schrieb Marius van den Beek:
>
> Hi everyone,
>>
>> it’s my first post to the list, so forgive me if I miss something obvious,
>> I tried to get all the information I could so if there was already a
>> discussion I’d be glad if you can point me towards this.
>>
>
>
> https://trello.com/c/fYB8fMY4/43-docker-in-galaxy-in-docker
> https://trello.com/c/tm1fPvyq
> https://trello.com/c/8k55T24F
> https://github.com/bgruening/galaxy-ipython
> https://github.com/bgruening/docker-galaxy-stable
>
> The galaxy-docker-stable image can be used to generate self-defined set of
> Tools with a full working Galaxy instance.
OK, thanks for the collection of resources, that's useful!
>
>
> I’m posting to start a discussion about how galaxy is going to handle
>> interactions with Docker.
>>
>> I know there are already some tools and resources out there that make use
>> of Docker. As I understood, we can already specify Docker as a job
>> destination in the tool xml, along with an image to use (See
>> https://github.com/apetkau/galaxy-hackathon-2014/tree/master/smalt for a
>> detailed example). This is cool, and allows for the controlling of
>> resources, but I think we could take the integration to the next step if
>> we
>> allowed the generation of Docker images in galaxy.
>>
>
>
> The Tool Shed can, since the latest release, generate you "ready to go"
> Dockerfiles with all your favorite tools.
That's pretty cool indeed, I didn't know this.
>
>
> I was thinking if there is a trusted baseimage (ie without root access), we
>> could let users install
>>
>
> Can you please elaborate on this a little bit more?
Sure. My idea was that the normal, non-admin galaxy user would be dropped
into a more or less populated container, with normal user rights, so
that there shouldn't be a way to do much harm to the docker host. Then they
could install their favorite R, python, perl, ... package (or import a
toolshed tool ...).
When satisfied and all software is there, a commit is triggered, and the
image would be stored for later use.
Or alternatively the steps necessary for installation would be transformed
into a script that is added and executed during the Docker build process.
The advantage here would be that you don't need to be admin to do this.
I'll try to get the basic functionality working, a demo of this might clear
things up.
>
>
> all the packages they need, commit after the install, and let them use this
>> new docker image. This could be beneficial for generating new tools with a
>> sandboxed version of the Galaxy Tool Factory (
>> http://www.ncbi.nlm.nih.gov/pubmed/23024011).
>>
>
>
> I am currently working on this (
>> https://bitbucket.org/mvdbeek/dockertoolfactory), and I would like to
>> have
>> your opinion on how to manage these user-generated Docker images.
>>
>> Some questions that I came across:
>>
>> Do we store the committed images in the user’s history (they could
>> potentially become very big!)?
>>
>> How to display available images to the user? Something like a per-user
>> image registry?
>>
>> How to identify available images (dataset_id as the tag?)
>>
>> How to transfer user-generated images to the toolshed?
>>
>> Is it a good idea at all to store user-generated images in the toolshed?
>>
>> What do we do if the security of the baseimage is compromised? Obviously
>>
>> we can blacklist execution of images, but what if somebody installed a
>> dangerous image from the toolshed, and is not aware of this?
>>
>
> To answer these questions it would be good to have some use-cases for your
> docker tool images.
> For example I think most of the time an IPython instance would be
> sufficient. You will only store notebook files in your history (small) and
> execute them on new datasets.
>
> Yes, especially since you can also interact with R and the shell inside
IPython, but right now it can't be used inside a workflow, for example ...
unless I missed something.
I would be really interested in your use-cases.
>
Well, in my experience, many biologists working with genomics data *can*
put together a commandline, but few are going to use galaxy,
because they can't run the specific tool that is not available in the
institute's instance, or they need a specific option not exposed ... . And
so the students/ other people in their team are also not going to use
galaxy.
I think the moment they had the possibility to run their code, I think a
lot of them could be lured in the galaxy ecosystem. If you show them how
easy it is to share their analyses with coworkers ... .
So you can let those people run any script. If it works as expected, it can
be used in workflows, and/or transformed in a (local) toolshed tool, that
would even allow version control.
Of course, a toolshed tool is not automatically a good tool, but it's
something to start with!
You could even think about l
