Thanks, Glyph!
I haven't tried this yet, but I think it is exactly what I need.
-- John Santos
On 8/28/2023 7:13 PM, Glyph wrote:
On Aug 28, 2023, at 2:41 PM, John Santos <j...@egh.com> wrote:
There is no "setup.py" How do I install from sources?
Pip is the only supported installer. If we want to support other installation
mechanisms, we need to add them to CI first.
I can't use pip for several reasons:
Luckily it's totally possible to use pip and satisfy all these requirements!
1) Customer systems are not Internet-connected and can't use pypi to retrieve
packages at install time.
Pip has an option, `--no-index`, which is explicitly for this sort of
non-internet-connected system.
For what it's worth, `setup.py` can and will /also/ use setuptools to reach out
to the internet to download stuff at install time; mostly, build dependencies.
It's running arbitrary code and sometimes that code will attempt to shell out
to easy_install.
2) Version control. I need to be sure the versions of all packages pulled in
due to dependencies exactly match what I'm using on my test/development systems.
A great deal of pip's infrastructure is dedicated to allowing for transitive
pinning of dependencies. You may want to look into pip-compile, specifically
with the --generate-hashes option: https://pypi.org/project/pip-tools/
<https://pypi.org/project/pip-tools/>
3) I need to have actual source code, not an opaque blob, for documentation
and accountability.
Twisted and all of its dependencies ship source distributions as well as
pre-built wheels to PyPI because many users have this requirement, so you don't
need to use the wheels.
The basic process for your type of build pipeline is two commands, although you
can get a lot fancier if you want.
The first command, you run on your development systems to build a meta-package
of all the source distributions that you use as input to your development
process; this needs Internet access and given your requirements this is the step
at which you'd unpack all the downloaded archives and have a look inside them to
review changes to validate updates for security, or whatever other purposes you
need the source code for:
1. pip download --no-binary :all: twisted hatchling flit_core wheel calver
setuptools_scm hatch_vcs hatch-fancy-pypi-readme
This command is annoyingly long because I am not sure how to tell `pip download`
to include all build dependencies to create a hermetic environment, but this is
the current transitive list of all build-deps for Twisted's dependency tree, as
far as I can tell. I think once everybody has adopted PEP 517 this will be simpler.
If you need twisted extras like [tls], [http2], etc, or you want to use this
process for a different project, the process of discovering these extra
build-time dependencies is basically just do this step, do the next step, look
for any package names in the installation error if you get one, then repeat.
Note that "--no-binary :all:" will tell Pip to download source distributions
only, for *all* packages. No prebuilt wheels, no binary components.
2. pip install --find-links . --no-index twisted
This will install Twisted entirely from the source distributions you just
downloaded with the previous command, no communication with PyPI at all, no
loading of blobs.
This is, of course, /much/ slower than using all the prebuilt stuff and caching
infrastructure one would normally invoke, it takes 1-2 minutes to get through
this full process as opposed to only a few seconds for a `pip install`, but it
satisfies your requirements.
I would suggest adding a few extra steps though, since install-time for
customers is quite different from verification-time for developers. Rather than
doing a flat install directly off the source distributions on customer systems,
I'd build your /own/ wheels from the source of dependencies you've validated
in-house, which would be something like:
# on your build system
pip wheel --find-links . --no-index twisted
cp *.whl /.../installer-media
# on customer system, later
pip install --no-index --find-links /.../installer-media/ twisted
Hope this helps!
-g
_______________________________________________
Twisted mailing list -- twisted@python.org
To unsubscribe send an email to twisted-le...@python.org
https://mail.python.org/mailman3/lists/twisted.python.org/
Message archived at
https://mail.python.org/archives/list/twisted@python.org/message/CX6H355S5DDXWDC4OYVF7PYLW4QR3MBE/
Code of Conduct: https://twisted.org/conduct
--
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
_______________________________________________
Twisted mailing list -- twisted@python.org
To unsubscribe send an email to twisted-le...@python.org
https://mail.python.org/mailman3/lists/twisted.python.org/
Message archived at
https://mail.python.org/archives/list/twisted@python.org/message/KTGN46KZBEF26QVJZXOYLD6RNB6G4POI/
Code of Conduct: https://twisted.org/conduct
