On Mon, Feb 22, 2016 at 9:02 PM, Nate Coraor <n...@bx.psu.edu> wrote: > On Mon, Feb 22, 2016 at 5:54 AM, Peter Cock <p.j.a.c...@googlemail.com> > wrote: >> >> Hi all, >> >> Last week at the Galaxy Admins video hangout Nate gave an >> overview of the way Python dependency handling is changing >> from using egg files to using wheel files instead - slide links at: >> >> ... >> >> Recently we've been trying to setup a replacement Galaxy instance >> where jobs are submitted to our SGE cluster as the linux user >> requesting the job. This has been a bumpy road, ... >> >> ... >> >> Clearly with Galaxy v16.01 the eggs and $PYTHON_EGG_CACHE >> will go away. What happens with wheels, Python virtual environments, >> and running jobs as the real user? > > > Hi Peter, > > Unlike eggs, which can be installed in their package format (as a single > .egg file) and used in place, wheels are unpacked and their contents reside > in the virtualenv's site-packages just as if you had `python setup.py > install`ed them, there's no cache directory. The "real user" should be able > to use Galaxy's virtualenv as long as the appropriate read/execute > permissions are set on a common group or other. > > Galaxy's virtualenv is passed down to the jobs when possible: > > https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/jobs/runners/util/job_script/DEFAULT_JOB_FILE_TEMPLATE.sh#L17 > > If you need to create a separate virtualenv for jobs to use, you can do this > with an <env> tag on the destination: > > > https://docs.galaxyproject.org/en/release_16.01/admin/framework_dependencies.html#galaxy-job-handlers > > There are instructions in that document on how to create your own Galaxy > virtualenv: > > > https://docs.galaxyproject.org/en/release_16.01/admin/framework_dependencies.html#managing-dependencies-manually > > I have not used the "real user" stuff in a very long time so I haven't > tested this, but I can't think of any major roadblocks that should prevent > it from working. > > --nate
Thanks Nate, this sounds promising - fingers crossed :) Maybe we should test the pre-release branch for v16.10 out? Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
