On Mon, Sep 23, 2013 at 1:25 PM, Björn Grüning <bjoern.gruen...@pharmazie.uni-freiburg.de> wrote: > Am Montag, den 23.09.2013, 12:57 -0500 schrieb John Chilton: >> On Mon, Sep 23, 2013 at 12:51 PM, Björn Grüning >> <bjoern.gruen...@pharmazie.uni-freiburg.de> wrote: >> > Hi Carlos! >> > >> >> Hi, >> >> >> >> I have created a tool that depends on biopython. I don't like the idea >> >> of having several copies of biopython around and also for best >> >> practice making workflows using my tool reproducible, I chose to make >> >> my package repository[1] dependent of package_biopython_1_62. >> >> >> >> [1]http://testtoolshed.g2.bx.psu.edu/view/cjav/package_ngs_tools_0_1_6 >> > >> > Cool thanks! I think that is best practise! >> >> I disagree strongly. This is not a good idea. My thought is if you are >> using a virtualenv, you shouldn't be trying to compose it with other >> python dependencies - to me the whole point of virtualenv is that it >> creates isolated little environments. I am not sure why duplicating >> the biopython install reduces reproduciblity. >> >> If you want to use individual Python packages using Galaxy's >> dependency mechanism, I think you should then package them up one at a >> time and hand modify PYTHONPATH and PATH - the way biopython is done. >> >> Galaxy should have a separate action to make this easy, say >> install_pip or something like that, as outlined by James Taylor at >> some point. > > Hi John, > > interesting point. What is the specific difference between venv and the > pip installation? In which case do you propose to use which method? > I never thought about different 'isolation-levels' (venv vs. the rest of > the toolshed). We need to document somehow the use cases if we have > different methods for python installations.
I would use the venv method for everything (unless there is some reason virtualenv cannot be used a certain case, I would not be surprised if I was over estimating its power) and it wouldn't be composable - let the broader Python community worry about Python dependencies, how to install them, how to package them, etc.... Galaxy should just be concerned with figuring out how to enable the environment before running a job. Let me know if I am being a jackass who is oversimplifying this and there is some reason it cannot be done. The use case for not using venv is not necessarily something I understand, but its clear you and Carlos and others want to have individual Python dependencies packaged in the tool shed the way Debian or CentOS do it. You will have to explain that use case to me :). While I don't like this approach, if it is something you guys want to do, you should have the tools to make it as easy as possible to do this. If that is something like setup_pip (i.e. wrapper for pip install) or setup_python (wrapper for python setup.py --prefix=$install_dir install), cool beans these should be pretty straight forward to implement and should save on some typing. I still think of them as anti-patterns though :). -John > > Ciao, > Bjoern > >> -John >> >> > >> >> I'm installing my package using the virtualenv option. As you can see >> >> below I marked biopython as prior_installation_required="True". >> >> However, I can see that virtualenv is installing a copy of biopython >> >> in 'venv' under my tool install(see below). This doesn't sound right. >> >> I was expecting that by making my repository dependent of biopython's >> >> one that source would be used in my tool. >> > >> > Can you try to insert that (maybe adopt, its not checked): >> > >> > <action type="set_environment_for_install"> >> > <repository name="package_biopython_1_62" owner="biopython"> >> > <package name="biopython" version="1.62" /> >> > </repository> >> > </action> >> > >> > Only with that the PYTHONPATH is populated. >> > >> >> My plan is to create any non-existing "Tool dependency definition" >> >> repository for every package my tool requires. This way you can always >> >> count the same version of any of these packages is being used. I >> >> thought that was the point, right? >> > >> > Yes! >> > >> >> Full ool_dependencies.xml: >> >> <?xml version="1.0"?> >> >> <tool_dependency> >> >> <package name="biopython" version="1.62"> >> >> <repository changeset_revision="ac9cc2992b69" >> >> name="package_biopython_1_62" owner="biopython" >> >> prior_installation_required="True" >> >> toolshed="http://testtoolshed.g2.bx.psu.edu"; /> >> >> </package> >> >> <package name="ngs-tools" version="0.1.6"> >> >> <install version="1.0"> >> >> <actions> >> >> <action type="setup_virtualenv">ngs-tools==0.1.6</action> >> >> </actions> >> >> </install> >> >> </package> >> >> </tool_dependency> >> >> >> >> Contents of site-packages in my tool install dir: >> >> $ ls -1 >> >> shed_tools_dependencies.central/ngs-tools/0.1.6/cjav/package_ngs_tools_0_1_6/3c646b8328bb/venv/lib/python2.7/site-packages/ >> >> Bio >> >> biopython-1.62-py2.7.egg-info >> >> BioSQL >> >> docopt-0.6.1-py2.7.egg-info >> >> docopt.py >> >> docopt.pyc >> >> easy-install.pth >> >> Levenshtein.so >> >> ngs >> >> ngs_tools-0.1.6-py2.7.egg-info >> >> pip-1.3.1-py2.7.egg >> >> python_Levenshtein-0.10.2-py2.7.egg-info >> >> setuptools-0.6c11-py2.7.egg >> >> setuptools.pth >> >> >> >> Thanks, >> >> Carlos >> >> ___________________________________________________________ >> >> Please keep all replies on the list by using "reply all" >> >> in your mail client. To manage your subscriptions to this >> >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >> >> >> >> To search Galaxy mailing lists use the unified search at: >> >> http://galaxyproject.org/search/mailinglists/ >> > >> > >> > >> > ___________________________________________________________ >> > Please keep all replies on the list by using "reply all" >> > in your mail client. To manage your subscriptions to this >> > and other Galaxy lists, please use the interface at: >> > http://lists.bx.psu.edu/ >> > >> > To search Galaxy mailing lists use the unified search at: >> > http://galaxyproject.org/search/mailinglists/ > > > ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
