On 02/05/2020 18:43, Mike Kelsey wrote:
Kenneth Hoste writes:
[ I asked about PIP pulling down setup.py dependencies: ]
This package contains the required setup.py, which itself has the argument,
install_requires=['datacat @
git+https://github.com/slaclab/datacat.git#subdirectory=client/python',
'requests',
'tqdm'],
I thought the |pip install| action took care of parsing this list, doing all
the requisite downloads internally, and leaving you with a fully functional
package with all of its dependencies satisfied.
We instruct pip to not auto-download-and-install missing dependencies
(which it does by default), because that often leads to installations
you can't reproduce later using the same easyconfig file.
The "pip check" is something we require in easyconfigs in our central
repository fairly recently, and we do so because we have learned the
hard way that stuff breaks if we don't ensure all required Python
packages are specified in the easyconfig file (see
https://github.com/easybuilders/easybuild-easyconfigs/issues/10462 for a
recent case of broken easyconfigs for exactly this reason).
Dependency hell is what one of my colleagues calls it :-)
For https://github.com/slaclab/datacat.git, you could either use
git_config (thanks a lot for your work on making that supported for
extensions as well!), or you could just download a source tarball
provided by GitHub
(https://github.com/slaclab/datacat/archive/stable.tar.gz for example),
Urgh? That's done automatically? I clearly don't know enough about GitHub.
Yes, for projects on GitHub you almost never really need to use the
git_config, you can grab a source tarball for tags and even individual
commits.
and use that. The 'subdirectory' part probably corresponds to the
'start_dir' easyconfig parameter?
I'll look into that as an option! What I ended up doing was ceating a
separate module for "datacat", and installing it as Python-only by copying
the needed subdirectory. Then I could specify it as an ordinary
dependence instead of exts_list.
That also works of course. :)
Whether that's the preferred option depends on whether it makes sense to
provide "datacat" as a separate installation (whether or not it's useful
by itself).
regards,
Kenneth
-- Mike
