[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
Fair enough. Pull request created: https://github.com/python/cpython/pull/16347 Regards, Vinay Sajip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IQGK5HFMV6ESBF7CS32U2BWVM23DIKXF/
[Python-Dev] Re: The Python 2 death march
On Wed, Sep 18, 2019, at 19:23, Kyle Stanley wrote: > Benjamin, what are you thoughts on usage of the "needs backport to 2.7" > label? For most of the PRs I've reviewed I tend to avoid adding it > myself, but I've seen it used periodically. It seems to be used rather > infrequently > (https://github.com/python/cpython/pulls?q=is%3Apr+label%3A%22needs+backport+to+2.7%22+is%3Amerged), > but I'm not entirely clear on when it should be used, particularly for > documentation-related PRs. > > Also, is there an official date when the label will be removed? > Removing the backport label seems to generally be at the discretion of > the release manager for that version, but I was curious as to whether > it would be removed on the sunset date or prior to then. Without the > label, backport PRs can still be opened manually of course, but > removing it would probably help to discourage 2.7 backports. I don't think the 2.7 backport label is currently overused. The 2.7 branch is already is a fairly quiet state. We've had less than 15 commits to 2.7 this month despite there being a week of more than 20 core developers intensely hacking on CPython. The label drives the automated backport tooling. There's no need to impede legitimate 2.7 backports. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/74TRPU6SE2B7WKBRJSEB4CNDFRMX33XC/
[Python-Dev] Re: The Python 2 death march
On Fri, Sep 13, 2019, at 18:18, Sumana Harihareswara wrote: > Hi. I've joined python-dev to participate in this thread (I don't have > email delivery turned on; I'll be checking back via the web). sorry :) > > Benjamin, I am sorry that I didn't check in with you, and assumed that > January 1, 2020 would be the the date of the final 2.7 point release. > (My understanding was based on Guido's EOL announcement from March last > year https://mail.python.org/pipermail/python-dev/2018-March/152348.html > -- I should have also gotten a review from you and not just the > Steering Council in https://github.com/python/steering-council/issues/14 > .) I'm going to continue this discussion here so I can make sure I > understand the policy decision properly, and then (if necessary) update > the FAQ. > > Based on what I've read here and what I see in > https://www.python.org/dev/peps/pep-0373/#maintenance-releases , it > sounds like the timeline will go something like: > > * 2019-10-19: release of 2.7.17 October > * October, November, and December 2019: developers continue to fix > issues in 2.7 > * 2020-01-01: code freeze for 2.7.18 release candidate > * January and February 2020: flexibility to fix any issues introduced > since the 2.7.17 release, but no other bugs or security issues, and no > 3.x backports Security issues will probably be fixed. At least, I wouldn't in abstract find that objectionable assuming someone wants to write a patch. > * ~2020-04-02: release candidate for 2.7.18 > * 2020-04-17: final 2.7.18 release I don't know if these will be the exact dates but probably close. > > Is this right? (If so, I can submit an update to PEP 373.) > > This is a little more complicated than I had anticipated when > communicating out about the sunsetting. But I can find a way either to > concisely communicate this, or to point to a user-friendly explanation > elsewhere. A succinct statement of the relevant information is: "After 10 years, the core developers of CPython are stopping development on the 2.7.x line. The last release will be in April 2020." If it's easier to communicate that the sunset of CPython 2 is April 2020, that seems fine with me. January 1 was a somewhat arbitrary date we put in the PEP when 2020 still seemed like a long way off but people wanted to know whether 2.7 would be released until 2021 or not. I was never going to make a 2.7 release literally on January 1. (Fighting with GPG would make short work of New Year's resolutions pertaining to temperance and strong language.) I failed to anticipate how strongly people would latch onto that exact moment in time as the end of Python 2. I additionally share the bemusement of some other commentators on this thread to the idea of Python 2 "support", which is not something ever promised to Python 2 (or 3) users by CPython core developers. Essentially, next year, we're changing our "support" policy of Python 2.7 from "none, but we're nice people" to "none". > > Thanks. > > -- > Sumana Harihareswara > Changeset Consulting > https://changeset.nyc > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/MXCGMTXDY7BX6JBBU36O5YFRWWBB3NQE/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ETOBPFVKKS5ZBIUAYLOBXFXPOZB7A357/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
> How easy would it be to search the sources and the docs for the ones that are currently public but not documented? Comparing the docs and .c files for functions that are not documented would be fairly trivial, but it's very difficult to define something as being public if it's not documented, since that's the main definition used. If it's not in the documentation, there's not a definitive way tell that it's public (AFAIK). The presence or lack of the name starting with an underscore can sometimes be an indicator, but that's not consistently reliable. On Mon, Sep 23, 2019 at 7:59 PM MRAB wrote: > On 2019-09-23 21:22, Vinay Sajip via Python-Dev wrote: > > OK - but that's just one I picked at random. There are others like it - > what would be the process for deciding which ones need to be made private > and moved? Should an issue be raised to track this? > > > It's not really a question of which ones should be made private, but of > which ones should be remain public. If it's mentioned in the docs, then > it's public, otherwise it should be private. > How easy would it be to search the sources and the docs for the ones > that are currently public but not documented? > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/4KUPJBNXXL7VD6JPAC7YCCTD56A5OUAZ/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U3WTOSLDLIZBIB4IJPJ4LSMWQNLY7W2V/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
On Mon, Sep 23, 2019 at 1:30 PM Vinay Sajip via Python-Dev wrote: > > OK - but that's just one I picked at random. There are others like it - what > would be the process for deciding which ones need to be made private and > moved? Should an issue be raised to track this? There are really two issues here: - hiding the symbols that *aren't* marked PyAPI_*, consistently across platforms. - finding symbols that are currently marked PyAPI_*, but shouldn't be. The first one is a pretty straightforward technical improvement. The second one is a longer-term project that could easily get bogged down in complex judgement calls. So let's worry about them separately. Even if there are too many symbols marked PyAPI_*, we can still get started on hiding all the symbols that we *know* should be hidden. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VLTZW3HQUK3ZAQEEKZIHCJRUOXRMUPVV/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
On 2019-09-23 21:22, Vinay Sajip via Python-Dev wrote: OK - but that's just one I picked at random. There are others like it - what would be the process for deciding which ones need to be made private and moved? Should an issue be raised to track this? It's not really a question of which ones should be made private, but of which ones should be remain public. If it's mentioned in the docs, then it's public, otherwise it should be private. How easy would it be to search the sources and the docs for the ones that are currently public but not documented? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4KUPJBNXXL7VD6JPAC7YCCTD56A5OUAZ/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
OK - but that's just one I picked at random. There are others like it - what would be the process for deciding which ones need to be made private and moved? Should an issue be raised to track this? Regards, Vinay Sajip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/USKL36S4SRDEK7TY46C2HIUHXFVRDQUQ/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
Le lun. 23 sept. 2019 à 21:36, Vinay Sajip via Python-Dev a écrit : > Nathaniel Smith wrote: > > Windows already has working symbol visibility handling, and PyAPI_FUNC is > > what controls it. So adding symbol visibility handling to Linux/macOS is > > just about making all the platforms consistent. There might be some weird > > choices being made, but I don't think you need to sort all those out as > > part of this. > > Well, _Py_DecodeLocaleEx is declared with PyAPI_FUNC, so would you expect it > to be exposed on Windows? _Py_DecodeLocaleEx() should not be used outside Python. It's really a low-level function which should not be used directly. Its definition should be moved to the internal API and it should use extern rather than PyAPI_FUNC(). It's a private API so we can make it internal. Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MEAAF636JYXNN7RKXJ4TKKUUBJALIMN4/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
Ah - I checked, and it's there OK ... (head scratch) Regards, Vinay Sajip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/POLJVZ7UJGQ74E5YYZ3G5HPNDLP6G5T5/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
Nathaniel Smith wrote: > Windows already has working symbol visibility handling, and PyAPI_FUNC is > what controls it. So adding symbol visibility handling to Linux/macOS is > just about making all the platforms consistent. There might be some weird > choices being made, but I don't think you need to sort all those out as > part of this. Well, _Py_DecodeLocaleEx is declared with PyAPI_FUNC, so would you expect it to be exposed on Windows? I haven't a Windows machine handy right now, but I would expect it not to be exposed, as it doesn't appear in PC/python3.def. I will check when I get a chance. Regards, Vinay Sajip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Q6H72YBNCKQYUQ4O4TMP65R4VEPEVLMT/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
Thanks for the pointer. According to that information, everything in Include/fileutils.h should be the portable public C API. But there are definitions in there which start with _Py - and the information relating to Include/cpython/*.h suggests that API prefixed with _Py is conventionally private. Is there a reason why (for example) _Py_DecodeLocaleEx appears in Include/*.h with a prefix conventionally suggesting it's private, and which is backed up by the fact that it's not mentioned in the C API documentation? If it's just there for the moment because no-one got around to moving it to Include/cpython/*.h, it would be useful to know that. I'm not trying to nit-pick here - if we're going to apply visibility rules, then things like this need to be absolutely clear. Just for info, I tried configuring and compiling with -fvisibility=hidden and declared default visibility on PyAPI_FUNC, PyAPI_DATA and PyMODINIT_FUNC, as well as one or two other declarations. The resulting build passes all tests run locally, except test_tools (which is currently also failing for me on master). I haven't yet pushed these changes to my public CPython fork, as it's still work in progress - which is why I'm asking these questions. Regards, Vinay Sajip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/G6P4RSVKFF7E6QYHBO4ZRLNHSNJWA6MT/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
On Mon, Sep 23, 2019, 08:28 Vinay Sajip via Python-Dev < python-dev@python.org> wrote: > > requires some newer tools like -fvisibility=hidden that work > > differently across different platforms, and so far no-one's done the > > work to sort out the details. > > I've started looking at this, but quite apart from the specifics of > applying -fvisibility=hidden, there are some things that aren't yet clear > to me about the intent behind some of our symbol definitions. For example, > the file Include/fileutils.h contains the definitions > > PyAPI_FUNC(wchar_t *) Py_DecodeLocale(const char *arg, size_t *size); > > and > > PyAPI_FUNC(int) _Py_DecodeLocaleEx(const char *arg, > wchar_t **wstr, > size_t *wlen, > const char **reason, > int current_locale, > _Py_error_handler errors); > > However, only the first of these is documented, though the definition via > PyAPI_FUNC implies that both are part of the public API. If this is the > case, why aren't both documented? If _Py_DecodeLocaleEx is not part of the > public API (and the leading underscore suggests so), should it be polluting > the symbol space? > > The comment for PyAPI_FUNC is "Declares a public Python API function and > return type". Is this really the case, or has PyAPI_FUNC been coopted to > provide external linkage for use by Python-internal code in different > compilation units? _Py_DecodeLocaleEx is called in > Modules/_testcapimodule.c and also in Objects/unicodeobject.c. > > If we want to take steps to restrict symbol visibility, it will > potentially affect all of the code base - so presumably, a PEP would be > required, even though it's an implementation detail from the point of view > of the language itself? > Windows already has working symbol visibility handling, and PyAPI_FUNC is what controls it. So adding symbol visibility handling to Linux/macOS is just about making all the platforms consistent. There might be some weird choices being made, but I don't think you need to sort all those out as part of this. -n ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TKSQQ5TLVRY4XYMSKVJ7X5GMXANOIDGG/
[Python-Dev] Bug in Bedevere related to detecting edited titles that add in the bpo number post-open
https://github.com/python/bedevere/issues/192 It should get fixed sometime today or at worst tomorrow. Until then, if someone edits a title after creating a PR to add an issue number, just add the `skip issue` label and then remove it again and it will run the check. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/T2CEVLYMNJCJMA23UX7SOJD72AMZNTAJ/
[Python-Dev] Bug in Bedevere related to detecting edited titles that add in the bpo number post-open
https://github.com/python/bedevere/issues/192 It should get fixed sometime today or at worst tomorrow. Until then, if someone edits a title after creating a PR to add an issue number, just add the `skip issue` label and then remove it again and it will run the check. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/M2JYBBXIMUQ5QAT2YHU6JSJY33C4GM3X/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
Vinay Sajip wrote: > > Right, I'm > > pretty sure that right now Python doesn't have any way to > > share symbols between .c files without also exposing them in the C > > API. > > On other C projects I've worked on, the public API is expressed in one set > of header files, and internal APIs that need to be exposed across modules are > described in > a different set of internal header files, and developers who incorrectly use > internal APIs > by including the internal headers could see breakage when the internals > change ... excuse > my naïveté, as I haven't done much at Python's C level - does this > discipline/approach not > apply to CPython? As of Python 3.8 we do this sort of separation: https://docs.python.org/3.8/whatsnew/3.8.html#build-and-c-api-changes. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GS2GJDU6J4RXWPHDB5UJJTHLVULOGVLX/
[Python-Dev] Re: The Python 2 death march
Stefan Behnel wrote: > Ned Batchelder schrieb am 10.09.19 um 16:54: > > this seems confusing to me > > What does the "official EOL date" mean if there's a release in April? It means we should all consider Python 2.7 EOL'ed come January 1 and Benjamin will make a release when it's convenient, and he just happens to know ahead of time that convenient for him is PyCon US 2020. :) > > Also, what day in April? For example, planning the release for the 1st > could possibly add further to the confusion. PyCon US 2020 is mid-April, so if the release is going to coincide with the conference then it won't be at the beginning of the month. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UW4N3TZZNWGKZQ7HPBSMQNFAAXL2TS2H/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
> requires some newer tools like -fvisibility=hidden that work > differently across different platforms, and so far no-one's done the > work to sort out the details. I've started looking at this, but quite apart from the specifics of applying -fvisibility=hidden, there are some things that aren't yet clear to me about the intent behind some of our symbol definitions. For example, the file Include/fileutils.h contains the definitions PyAPI_FUNC(wchar_t *) Py_DecodeLocale(const char *arg, size_t *size); and PyAPI_FUNC(int) _Py_DecodeLocaleEx(const char *arg, wchar_t **wstr, size_t *wlen, const char **reason, int current_locale, _Py_error_handler errors); However, only the first of these is documented, though the definition via PyAPI_FUNC implies that both are part of the public API. If this is the case, why aren't both documented? If _Py_DecodeLocaleEx is not part of the public API (and the leading underscore suggests so), should it be polluting the symbol space? The comment for PyAPI_FUNC is "Declares a public Python API function and return type". Is this really the case, or has PyAPI_FUNC been coopted to provide external linkage for use by Python-internal code in different compilation units? _Py_DecodeLocaleEx is called in Modules/_testcapimodule.c and also in Objects/unicodeobject.c. If we want to take steps to restrict symbol visibility, it will potentially affect all of the code base - so presumably, a PEP would be required, even though it's an implementation detail from the point of view of the language itself? Regards, Vinay Sajip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/H726K4PDW6PI34VVBX6HN26MAE5IZUNG/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
> Moving some of the especially common identifier references into the > interpreter state struct may make sense. > Adding more process globals wouldn't be desirable though, as they're one of > the more common ways of breaking encapsulation between subinterpreters > (hence Eric's efforts to eliminate as many of them as he reasonably can). Well, another way of breaking encapsulation is by having ad hoc statics (as opposed to globals) which should really be in the interpreter state - it prevents compartmentalizing the GIL into per-subinterpreter GILs, for example. That's my motivation for looking at this area - I spent a bit of time working with Eric at the recent core dev sprint, and wanted to explore the problems in this area. Regards, Vinay ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/43BYLJIZXPTVBGZTTQUTSPICZVWIF6HR/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
On Sat., 21 Sep. 2019, 5:28 am Vinay Sajip via Python-Dev, < python-dev@python.org> wrote: > I just ran an analysis of static variable definitions in CPython code, > using clang, on Ubuntu and Windows. The results should be available here: > > https://cpython.red-dove.com/ > > As I understand it, _Py_IDENTIFIER instances are supposed to hold constant > strings that are used in Python - e.g. "__class__", "__dict__" and so on. I > noticed that there are numerous duplicates of these - e.g. 8 instances of > __name__, 11 instances of __dict__, and so on - each instance is defined as > static in its source file and so completely distinct from the others. > > I realise the overall amount of memory used by these structures isn't > large, but is there any particular benefit to having these multiple copies? > The current situation seems a little untidy, at least. What would be the > disadvantage of making them extern in the headers and allocating them once > in some consts.c module? After all, they seem for the most part to be > well-known constant strings that don't need to be encapsulated in any > particular C compilation unit. > Moving some of the especially common identifier references into the interpreter state struct may make sense. Adding more process globals wouldn't be desirable though, as they're one of the more common ways of breaking encapsulation between subinterpreters (hence Eric's efforts to eliminate as many of them as he reasonably can). Cheers, Nick. > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IVMLDHZOE2WCGYA5PZQ24R3PYBQEUVXH/
