On Thu, 30 Aug 2018 at 23:13 Nathaniel Smith <n...@pobox.com> wrote:
> On Thu, Aug 30, 2018 at 6:52 PM, Brett Cannon <br...@python.org> wrote:
> >
> >
> > On Thu, 30 Aug 2018 at 11:21 Nathaniel Smith <n...@pobox.com> wrote:
> >>
> >> If we're going to rethink this,
> >
> >
> > Well, I didn't want to "rethink" so much as "fill in". :)
> >
> >>
> >> then I would really like to move away from assigning special meaning to
> >> specific combinations of tags. The thing where if you use cp36m as you
> >> Python ABI tag then you're forced to use cp36 as your python dialect tag
> >> doesn't make sense. And the thing where if you have a wheel with an
> >> extension module tagged as cp36-abi3, then that works fine on 3.7,
> >
> >
> > I wouldn't expect that to if the stable ABI was expanded in Python 3.7
> and
> > you used the expanded part.
>
> In my example, you have a wheel using the 3.6 ABI, running on 3.7.
> That's case that's supposed to work :-).
>
> >>
> >> but if you *remove* the extension module then it *stops* working on 3.7?
> >> That's just bizarre...
> >
> >
> > I don't quite follow what you mean by "remove the extension module then
> it
> > stops". What stops by removing the extension module(s)?
>
> If I'm reading your proposal right, it says that when running on
> Python 3.7, pip should be willing to install wheels tagged cp36-abi3,
> but not wheels tagged cp36-none. But conceptually, if you take the
> extension modules out of a cp36-abi3 wheel, then you're left with a
> cp36-none wheel. So this is weird. Anywhere we're willing to install a
> cp36-abi3 wheel, we should also be willing to install a cp36-none
> wheel.
>
Sure. I took it out because it complicated the logic in my head and only 7
projects have such a wheel.
>
> >>
> >>
> >> So my suggestions:
> >>
> >> * Make the 3 tag categories totally independent. Compute a separate set
> >> for each, and then take the full cross product.
> >
> >
> > I think Paul was hinting at this as part of his "wildcard" idea (and I
> > honestly thought of this initially as well as it greatly simplifies
> things).
> > So what would cp36-cp36m-plat expand to?
>
> I don't know what it means to "expand" a wheel tag. Are you punning
> the tag as also describing a Python installation ("CPython 3.6, with a
> certain soabi tag and platform"), and then asking to find all the
> wheel tags that could go in that installation?
Potentially because this comes up if you're trying to create a
multi-platform lock file for dependencies or you want to download the
dependencies for your cloud deployment which differs from your dev machine.
> You can't actually do
> this in general – for example, determining whether a target
> interpreter is compatible with manylinux wheels requires running some
> special sniffing code on that interpreter.
>
Maybe not entirely in general, but you can come pretty close.
>
> > cp36: cp3N where N is any positive digit(s)? And then toss in py3 and
> py3N?
> > Prefer exact, then generic '3', then older, and finally newer?
>
> Given that we don't have any real use cases for cpXY and pyXY, I'd
> rather not expand the options – just preserve what we do now... so for
> CPython 3.6, I'd say py3, py3N for N <= 6, cp3N for N <= 6, and I
> don't really care about the exact order – some sort of
> more-specific-before-less-specific makes sense, and whatever we do now
> is probably fine. If someone goes wild and starts distributing py35
> and py36 wheels for the same package (via a "36to35" tool, I guess?)
> then that's probably what you want? But I don't imagine this will ever
> be an important use case. We tried it with 2to3, and everyone decided
> they'd rather write in the subset language for a decade instead.
>
> > cp36m: that, abi3, and then 'none'? Do we care if someone has some crazy
> ABI
> > that no one understands like 'b' (and if so should it only be applied to
> > 'py' interpreter versions which break your nice cross product
> simplicity)?
> > plat: depends on platform.
>
> Do you mean, what happens if we find ourselves running on an
> interpreter we don't recognize (not cpython/pypy/jython/...), and that
> interpreter returns something wacky from
> sysconfig.get_config_var("SOABI")?
That and people who have weird ABIs for CPython like 'b' as I found on PyPI.
It's hard to get a clear signal of what people want because people seem to
be signalling a desire to be fairly permissive, but then people bring up
examples where they don't. ;)
> I think there's a reasonable
> argument that in that case we should only accept 'none' as the tag.
> (And then maybe whoever invented this new interpreter gets the job of
> adding sysconfig.get_supported_abi_tags() as a standard stdlib feature
> :-).)
>
> > And the match preference goes to platform, interpreter version, and then
> > ABI? I think that would work in terms of ignoring C ABIs that have very
> > little chance of linking while being the broadest in terms of accepting a
> > wheel that has some semblance of a chance of running.
>
> I don't think the preference order matters that much as long as its
> well-defined. The only cases where it would affect things are like...
>
> Suppose in the future we add support for platform tags based on
> different ISA levels, like x86_64_avx2 vs x86_64_sse3 vs x86_64. Then
> you could find yourself in a situation where examplepkg v1.2.3 has the
> following wheels available:
>
> cp36-cp36m-manylinux1_x86_64
> cp36-abi3-manylinux1_x86_64_avx2
>
> Our environment is CPython 3.6, running on a manylinux1-compatible OS
> and we do have AVX2 support available, so either of these wheels could
> work. The first wheel has a "better" ABI (cp36m > abi3), but a "worse"
> platform (x86_64 < x86_64_avx2). So which one should we pick?
>
> I have zero intuition here. Whoever compiled these wheels is clearly
> perverse, and they should stop doing silly things like this.
>
> > And how would that apply to PyPy3? Same ranging on 'pp36N` and drop the
> > 'abi3' insertion?
> >
> >>
> >>
> >> * Since the stable ABI actually changes over time, we should define new
> >> tags abi35, abi36, etc. that mean "requires the stable ABI as defined by
> >> this version of cpython or higher", instead of relying on abi3 + a
> dialect
> >> tag. (Imagine if pypy started implementing the stable ABI –we'd have to
> >> start allowing cpXY tags to match PyPy.)
> >>
> >> * Plan to move away from the pyXY and cpXY tags over time; they're
> >> confusing and not useful. Of course this will have to be a gradual
> process,
> >> but if pip stops requiring them now then in a year or two we could make
> >> setuptools stop generating then.
> >
> >
> > What would you do then about preferred match order for pure Python
> wheels?
> > E.g. how do you preferably match against Python 3.7 wheels over 3.6 when
> > running a Python 3.7 interpreter? Or are you suggesting equivalent ABI
> tags
> > to make up for the pyXY tags and the interpreter tag simply gets ignored?
>
> I don't understand your questions, sorry :-(.
>
> I'm just saying: we should make the stable-ABI tags self-contained, so
> that 'py3-abi36' would be sufficient to express "uses the stable abi,
> as it existed on python 3.6". This would match the same set of
> interpreters that that 'cp36-abi3' is currently used to target. And
> also, we should try to gradually decrease the usage of pyXY and cpXY
> tags (e.g. eventually setuptools should stop generating them by
> default).
>
Your comment said "move away from pyXY and cpXY", which suggested to me
that you were wanting to drop the first part of the tag triple, not simply
downgrade it to 'py3', so just ignore my questions based on confusion. :)
-Brett
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/6MTPUSPFPSCI7RH4YA2SGAD3ESKUSZFX/
