Yes, it seems like a number of the ABI3 prerequisites have caught up since 2016. Now is a great time to revisit the issue.
On Tue, Jul 17, 2018 at 4:29 PM Shaun Walbridge <swalbri...@esri.com> wrote: > This is no longer accurate -- since early 2017, the Windows Python builds > include python3.dll alongside the major.minor DLL. > > On 7/17/18, 4:24 PM, "Phil Thompson" <p...@riverbankcomputing.com> wrote: > > On 17 Jul 2018, at 8:57 pm, Daniel Holth <dho...@gmail.com> wrote: > > > > According to a link in this chain, virtualenv lately copies > python3.dll (+ python3.7 e.g.) on Windows. So hopefully the extension links > with that dll. > > Yes. Historically at least, Anaconda does not include that DLL and > (IMHO) is inherently broken. > > Phil > > > On Tue, Jul 17, 2018, 15:51 Nathaniel Smith <n...@pobox.com> wrote: > > The promise behind the limited ABI is exactly that if your extension > works on 3.x, it will also work on 3.y, for y >= x. > > > > One thing to watch out for: normally extension modules on Linux and > MacOS don't try to link to libpython. Instead they trust that someone else > will make sure they only get loaded into a compatible python interpreter, > and that all the symbols they need from python will be injected by the host > process. > > > > On Windows, the way the dynamic loader works, you can't do this: > every extension module has to explicitly say "I'm getting PyNumber_Add from > the dll named: somethingsomething.dll" > > > > But traditionally, version X.Y of python includes a pythonXY.dll, so > there's no consistent name across releases. So even if your library uses > only the limited ABI and all of your imports could just as well come from > python36.dll or python37.dll... you need some way to express that, and only > Windows has this problem. > > > > I feel bad sending this without doing my own research, but I don't > have access to a Windows box right now. Does anyone know if this problem > has been solved? Is it still true that Windows python dlls always include > the python version in their name? > > > > -n > > > > On Tue, Jul 17, 2018, 09:38 Paul Moore <p.f.mo...@gmail.com> wrote: > > On 17 July 2018 at 16:59, Cosimo Lupo <cos...@anthrotype.com> wrote: > > > I would like to revive this 5 year old thread and see if we can > stir things > > > up a bit. > > > > > > Basically the problem is that, in the current state of the PEPs > and build > > > tools, it is still not possible to build and distribute a Windows > wheel that > > > includes an extension module compiled with Py_LIMITED_API. > > > Setuptools can successfully build such extension module on Windows > (with > > > `.pyd` file extension and no extra specifiers in the module > filename), and > > > these seems to work at least on CPython 3.5 and above. However the > > > `--py-limited-api cpXX` option of bdist_wheel command fails on > Windows > > > because it attempts to use the `abi3` tag but the latter is not in > the list > > > of compatible tags for that platform. > > > One can work around this by creating a wheel with `none` as the > abi tag, and > > > `cp35.cp36.cp37` as the python implementation tag but this feels a > bit > > > hackish. > > > > > > Here are some unresolved questions from the old distutils-sig > thread, > > > quoting Paul Moore: > > > > > >> 2. How should tools determine which ABIs a given Python supports? > > >> (This is the get_supported function in wheel and distlib). The > "Use" > > >> section of the PEP ( > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.python.org_dev_peps_pep-2D0425_-23id1&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=CI-50dMn_QF7jh6LxyY6q37nHnL8Zn0dP_GnjnXZ6iE&e= > ) > > >> gives a Linux-based example, but nothing normative and nothing > that is > > >> understandable to a Windows user. > > > > > > And from Vinay Sajip > > > > > >> For Windows, we (eventually) need some low-level > sysconfig-supported way > > >> to get the ABI information in an analogous way to how it happens > on POSIX: > > >> and > > >> because that's not currently there, distlib doesn't provide any > ABI > > >> information > > >> on Windows other than "none". > > > > > > Other related links: > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_pypa_pip_issues_4445&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=FbWIuKimM7I-r1z0drqZNQuxpypWBonCVRz9Jc8ryJ8&e= > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_pipermail_distutils-2Dsig_2018-2DJanuary_031856.html&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=c5RVqW1Zda4NUInIER9iAEAH3fFfeBSvoOkf_KSKtsw&e= > > > > > > So.. what needs to be done here to allow distributing/installing > Windows > > > wheels with Py_LIMITED_API support? > > > > IMO, the question I posed back then remains key. Vinay's response is > > fair, but I don't think that waiting for core Python to provide > > something via sysconfig is practical (it's not happened yet, so why > > would we expect things to change?). So I think the next step is > > probably for someone to propose an algorithm that can be used. > > Actually, what I'd like to see is a full end to end proposal of the > > process someone would use to build and install a limited-ABI > > extension. That would probably tease out a number of issues. > > > > I imagine the steps would be something like this: > > > > 1. Create an extension. Presumably you'd need to #define > > PY_LIMITED_ABI in the source. > > 2. Build a wheel from it - how would you do that? I gather it's > > possible to do this with plain setuptools - would it be necessary to > > do this with setuptools/bdist_wheel, or should there be a way to > > request a limited ABI build via pip? If we do want to be able to > > request this from a build tools like pip, do we need something in PEP > > 517? Are we only looking at the prebuilt wheel case, or do we need to > > support building from source? > > 3. What tags would the built wheel have? > > 4. Install that wheel - `pip install xxx`. Pip needs to be able to > > enumerate the full list of valid tags here (cp37-abi3, cp3-abi3, ...) > > There are also questions like - if there's a limited ABI wheel and a > > full ABI (version specific) wheel, which takes precedence? > > > > I don't honestly know how well the limited ABI actually achieves its > > goals - is "cp3-abi3-win_x86_64" a realistic tag to apply? Can > limited > > ABI wheels really be used on any version of Python 3? That's a > > question for python-dev, rather than distutils-sig, but if we take > the > > position that this is what's promised, and it later turns out not to > > be true, we've got a lot of wheel renaming to do when Python 3.10 > > comes out and it doesn't support the same limited ABI as 3.x for x < > > 10... > > > > Also, does the limited ABI work on other platforms? If it does, we > > should ensure that the Windows support works the same. If it doesn't, > > do we want a Windows-only solution (why is that OK?) or should we > > extend to (say) manylinux or OSX (at the risk of making the job too > > big for anyone to actually get anywhere with it). > > > > So to move this forward, I think someone needs to write up the > process > > of building and using a limited ABI wheel, as described above, and > > document suggested answers to the various questions that will come > out > > in the process of going through the details. Is that something you'd > > be willing to take on? > > > > From that, we'd have something concrete to debate. I'm not sure how > > many people have an interest in this topic, so getting people with > the > > necessary knowledge to chime in might take some work (I'm interested, > > but I don't have detail understanding of build options and linking > > conventions). The ultimate goal would be some sort of PEP covering > > handling limited ABI extensions within the packaging infrastructure. > > > > Does that seem reasonable? Is that the sort of guidance you were > > looking for? I doubt anything is going to happen on this subject > until > > someone with the interest in moving it forward steps up to do the > work > > of making a proposal and collecting community views. > > > > Paul > > -- > > Distutils-SIG mailing list -- distutils-sig@python.org > > To unsubscribe send an email to distutils-sig-le...@python.org > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_mailman3_lists_distutils-2Dsig.python.org_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=M8BIYDHADRC2DG3p_8p4E-jtXEvvKIrEAFpzMVko0is&e= > > Message archived at > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_archives_list_distutils-2Dsig-40python.org_message_F3QCJTGKFZJFWWV4CLLQFQ6XGBAQNNFX_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=ZwMkKEX6br2nQj9KrQhvui3DANLGlR32_BOwALjytVw&e= > > -- > > Distutils-SIG mailing list -- distutils-sig@python.org > > To unsubscribe send an email to distutils-sig-le...@python.org > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_mailman3_lists_distutils-2Dsig.python.org_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=M8BIYDHADRC2DG3p_8p4E-jtXEvvKIrEAFpzMVko0is&e= > > Message archived at > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_archives_list_distutils-2Dsig-40python.org_message_YXMNZEF4DGCNFFE3M66LVBOUT73HCPTF_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=b6Ex8zkpUUKFFdzBdGn9uEVciRHgK5a8RMHLccIB9dk&e= > > -- > > Distutils-SIG mailing list -- distutils-sig@python.org > > To unsubscribe send an email to distutils-sig-le...@python.org > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_mailman3_lists_distutils-2Dsig.python.org_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=M8BIYDHADRC2DG3p_8p4E-jtXEvvKIrEAFpzMVko0is&e= > > Message archived at > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_archives_list_distutils-2Dsig-40python.org_message_H2DHMDLYA77BDGDRYLFEFY6ZFS7QVNWI_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=j402Afyg9SFbc2UK-S8ZlUlw8SDTncSNL9Ais3t-ISY&e= > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_mailman3_lists_distutils-2Dsig.python.org_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=M8BIYDHADRC2DG3p_8p4E-jtXEvvKIrEAFpzMVko0is&e= > Message archived at > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mm3_archives_list_distutils-2Dsig-40python.org_message_UORLWC6PNWHKOCL4TO5ZAXZD5HTPAU6J_&d=DwICAg&c=n6-cguzQvX_tUIrZOS_4Og&r=YFaRLkcUCdDkLrpTbNOUV9J1CwYBCTMwgm5tdQkRSm4&m=xlQFE7IsUpY_J8xwu4_W--sNGrS38xw5c9B0ZJmewgE&s=NLh87pQi9YHUUWCZtAma8wk6VHqgd93f9wIPVfUEubw&e= > > >
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/AYKACC4TKFWWJI4FBOATGE7PJ2EIP6TN/
