On 17 Jul 2018, at 8:57 pm, Daniel Holth <dho...@gmail.com> wrote: > > According to a link in this chain, virtualenv lately copies python3.dll (+ > python3.7 e.g.) on Windows. So hopefully the extension links with that dll.
Yes. Historically at least, Anaconda does not include that DLL and (IMHO) is inherently broken. Phil > On Tue, Jul 17, 2018, 15:51 Nathaniel Smith <n...@pobox.com> wrote: > The promise behind the limited ABI is exactly that if your extension works on > 3.x, it will also work on 3.y, for y >= x. > > One thing to watch out for: normally extension modules on Linux and MacOS > don't try to link to libpython. Instead they trust that someone else will > make sure they only get loaded into a compatible python interpreter, and that > all the symbols they need from python will be injected by the host process. > > On Windows, the way the dynamic loader works, you can't do this: every > extension module has to explicitly say "I'm getting PyNumber_Add from the dll > named: somethingsomething.dll" > > But traditionally, version X.Y of python includes a pythonXY.dll, so there's > no consistent name across releases. So even if your library uses only the > limited ABI and all of your imports could just as well come from python36.dll > or python37.dll... you need some way to express that, and only Windows has > this problem. > > I feel bad sending this without doing my own research, but I don't have > access to a Windows box right now. Does anyone know if this problem has been > solved? Is it still true that Windows python dlls always include the python > version in their name? > > -n > > On Tue, Jul 17, 2018, 09:38 Paul Moore <p.f.mo...@gmail.com> wrote: > On 17 July 2018 at 16:59, Cosimo Lupo <cos...@anthrotype.com> wrote: > > I would like to revive this 5 year old thread and see if we can stir things > > up a bit. > > > > Basically the problem is that, in the current state of the PEPs and build > > tools, it is still not possible to build and distribute a Windows wheel that > > includes an extension module compiled with Py_LIMITED_API. > > Setuptools can successfully build such extension module on Windows (with > > `.pyd` file extension and no extra specifiers in the module filename), and > > these seems to work at least on CPython 3.5 and above. However the > > `--py-limited-api cpXX` option of bdist_wheel command fails on Windows > > because it attempts to use the `abi3` tag but the latter is not in the list > > of compatible tags for that platform. > > One can work around this by creating a wheel with `none` as the abi tag, and > > `cp35.cp36.cp37` as the python implementation tag but this feels a bit > > hackish. > > > > Here are some unresolved questions from the old distutils-sig thread, > > quoting Paul Moore: > > > >> 2. How should tools determine which ABIs a given Python supports? > >> (This is the get_supported function in wheel and distlib). The "Use" > >> section of the PEP (http://www.python.org/dev/peps/pep-0425/#id1) > >> gives a Linux-based example, but nothing normative and nothing that is > >> understandable to a Windows user. > > > > And from Vinay Sajip > > > >> For Windows, we (eventually) need some low-level sysconfig-supported way > >> to get the ABI information in an analogous way to how it happens on POSIX: > >> and > >> because that's not currently there, distlib doesn't provide any ABI > >> information > >> on Windows other than "none". > > > > Other related links: > > https://github.com/pypa/pip/issues/4445 > > https://mail.python.org/pipermail/distutils-sig/2018-January/031856.html > > > > So.. what needs to be done here to allow distributing/installing Windows > > wheels with Py_LIMITED_API support? > > IMO, the question I posed back then remains key. Vinay's response is > fair, but I don't think that waiting for core Python to provide > something via sysconfig is practical (it's not happened yet, so why > would we expect things to change?). So I think the next step is > probably for someone to propose an algorithm that can be used. > Actually, what I'd like to see is a full end to end proposal of the > process someone would use to build and install a limited-ABI > extension. That would probably tease out a number of issues. > > I imagine the steps would be something like this: > > 1. Create an extension. Presumably you'd need to #define > PY_LIMITED_ABI in the source. > 2. Build a wheel from it - how would you do that? I gather it's > possible to do this with plain setuptools - would it be necessary to > do this with setuptools/bdist_wheel, or should there be a way to > request a limited ABI build via pip? If we do want to be able to > request this from a build tools like pip, do we need something in PEP > 517? Are we only looking at the prebuilt wheel case, or do we need to > support building from source? > 3. What tags would the built wheel have? > 4. Install that wheel - `pip install xxx`. Pip needs to be able to > enumerate the full list of valid tags here (cp37-abi3, cp3-abi3, ...) > There are also questions like - if there's a limited ABI wheel and a > full ABI (version specific) wheel, which takes precedence? > > I don't honestly know how well the limited ABI actually achieves its > goals - is "cp3-abi3-win_x86_64" a realistic tag to apply? Can limited > ABI wheels really be used on any version of Python 3? That's a > question for python-dev, rather than distutils-sig, but if we take the > position that this is what's promised, and it later turns out not to > be true, we've got a lot of wheel renaming to do when Python 3.10 > comes out and it doesn't support the same limited ABI as 3.x for x < > 10... > > Also, does the limited ABI work on other platforms? If it does, we > should ensure that the Windows support works the same. If it doesn't, > do we want a Windows-only solution (why is that OK?) or should we > extend to (say) manylinux or OSX (at the risk of making the job too > big for anyone to actually get anywhere with it). > > So to move this forward, I think someone needs to write up the process > of building and using a limited ABI wheel, as described above, and > document suggested answers to the various questions that will come out > in the process of going through the details. Is that something you'd > be willing to take on? > > From that, we'd have something concrete to debate. I'm not sure how > many people have an interest in this topic, so getting people with the > necessary knowledge to chime in might take some work (I'm interested, > but I don't have detail understanding of build options and linking > conventions). The ultimate goal would be some sort of PEP covering > handling limited ABI extensions within the packaging infrastructure. > > Does that seem reasonable? Is that the sort of guidance you were > looking for? I doubt anything is going to happen on this subject until > someone with the interest in moving it forward steps up to do the work > of making a proposal and collecting community views. > > Paul > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/F3QCJTGKFZJFWWV4CLLQFQ6XGBAQNNFX/ > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/YXMNZEF4DGCNFFE3M66LVBOUT73HCPTF/ > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/H2DHMDLYA77BDGDRYLFEFY6ZFS7QVNWI/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/UORLWC6PNWHKOCL4TO5ZAXZD5HTPAU6J/
