Thanks for that feedback. Looks like RECORD is the one to use.
The names of the directories ending in dist-info seem to be uniformly:
package-version.dist_info
but the directory names associated with eggs come in a lot of flavors:
anndata-0.6.19-py3.6.egg
cutadapt-2.10.dev20+g93fb340-py3.6-linux-x86_64.egg
scanpy-1.5.2.dev7+ge33a2f33-py3.6.egg
h5py-2.9.0-py3.6-linux-x86_64.egg
simplejson-3.17.0-py3.6.egg-info
johnnydep does not give any hints that this is coming:
johnnydep --output-format pinned h5py
#relevant part: h5py==2.10.0
What would be some small examples for other package managers, I would
like to see what they have as equivalents to dist-info and egg-info so
that the script does not choke on it.
Some progress with the test script. It can now convert a virtualenv
to a regular directory
and migrate the site-packages contents to a shared area. A second
migration of a copy of the same virtualenv to a different regular
directory correctly makes links to the first set.
(That is, two normal directories both linked to one common set of
packages.) And the test program (johnnydep) runs in both with
PYTHONPATH set correctly. But preinstalling, that is setting links to
the common directory before doing a normal install is tricky because
of the name inconsistencies. To do that it must run johnnydep to get
the necessary information, and that is not very fast. A normal
install of johnnydep itself, complete with downloads, takes less time
than that programs own analysis!
time johnnydep johnnydep
#21s
vs.
rm -rf ~/.cache/pip #force actual downloads
#too fast to measure
time python3 -m venv johnnydep
#2.3s
source johnnydep/bin/activate
#too fast to measure
time python -m pip install -U pip #update 9.0.3 to 20.1.1
#3.4s
time pip3 install johnnydep
#7.8s
Probably a package with a huge amount of compilation would be a win
for a preinstall, but
it is at this point definitely not an "always faster" option.
Thanks,
David Mathog
On Fri, Jun 26, 2020 at 2:51 PM John Thorvald Wodder II
<jwod...@gmail.com> wrote:
>
> On 2020 Jun 26, at 15:50, David Mathog <dmat...@gmail.com> wrote:
>
> > Still, how common is that? Can anybody offer an estimate about what
> > fraction of packages use different names like that?
>
> Scanning through the wheelodex.org database (specifically, a dump from
> earlier this week) finds 32,517 projects where the wheel DOES NOT contain a
> top-level module of the same name as the project (after correcting for
> differences in case and hyphen vs. underscore vs. period) and 74,073 projects
> where the wheel DOES contain a module of the same name. (5,417 projects
> containing no modules were excluded.) Note that a project named "foo-bar"
> containing a namespace package "foo/bar" is counted in the former group.
>
> Of the 32,517 non-matching projects, 7,117 were Odoo projects with project
> names of the form "odoo{version}_addon_{foo}" containing namespace modules of
> the form "odoo/addons/{foo}", and 3,175 were Django projects with project
> names of the form "django_{foo}" containing packages named just "{foo}". No
> other major patterns seem to stand out.
>
> -- John Wodder
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/V445KCPLKMEVSSEAKX776DMNSPL76JRR/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at
https://mail.python.org/archives/list/distutils-sig@python.org/message/TT6WZTTBMEWHTZD56HXH42JFKEI5VECK/
