[Python-Dev] Re: urllib: addressing inflexibility in scheme-based joining
On Thu, Feb 10, 2022 at 10:23:59PM -0700, Lincoln Auster wrote: > This is a follow-up RFC on PR #30520 (BPO 46337) with regard to urllib's ... > It's been about a month since I wrote that PR, and it was marked stale a > day or two ago. Would anyone be willing to give it a look for feedback > and a potential merge? Thanks for bringing this up. https://github.com/python/cpython/pull/30520 I will review it. If any other core-dev reviews earlier than me, I am good with it too. Thank you, Senthil ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/T73EN4OG5FOC53F2NHFSBDUJQMKJPA5X/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] urllib: addressing inflexibility in scheme-based joining
Hi, This is a follow-up RFC on PR #30520 (BPO 46337) with regard to urllib's potentially inflexible scheme-based URL manipulation. In my ticket, I proposed supplementing the hard-coded uses_* scheme lists that control URL parsing and joining behavior with an optional enum set that may override behavior when calling a function that would require it. (More detail is in that BPO and code is obviously in that PR.) It's been about a month since I wrote that PR, and it was marked stale a day or two ago. Would anyone be willing to give it a look for feedback and a potential merge? As a side note, I was recently made aware of the bit of an existential crisis wrt urllib. Without addressing it directly, I would like to say that my PR is small, backwards compatible, and focused exclusively on URL parsing, which (at least in my reading of the relevant thread) is still considered to be useful if a bit fragile. I wouldn't consider my PR to be in conflict with any ongoing scope reconsideration (though deprecation is, of course, another matter entirely). Thanks for any feedback and review! -- Lincoln Auster They/Them ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NP7GK3IEB3W6OJ7KAC2GSSMAT6WREIQF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?
On 2022-02-10 2:58 p.m., Victor Stinner wrote: Would it make sense to move the pythoncapi_compat project under the GitHub Python or PSF organization to make it more "official" and a little bit more sustainable? I think that makes sense. Extensions typically have this kind of compatibility code built into them, so they can support multiple Python versions. It makes more sense to centralize that code. It will be easier to keep up-to-date and will be better quality. I think having the project more tightly associated with CPython is good too. When a change is made to the CPython extension API, it would be good to consider how pythoncapi_compat will need to be updated. E.g. how can extensions that want to support both old and new versions of the API work? Regards, Neil ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/L62AAY6JZIYIRD7FYMC567XFM5PBAIZK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?
I created a poll on Discourse: https://discuss.python.org/t/move-the-pythoncapi-compat-project-under-the-github-python-or-psf-organization/13651 It will be closed automatically in 10 days. Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KJ32YNASRWXX7SEY7T6HVMSIGV2DQJC7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Move the pythoncapi_compat project under the GitHub Python or PSF organization?
Hi, Would it make sense to move the pythoncapi_compat project under the GitHub Python or PSF organization to make it more "official" and a little bit more sustainable? "The pythoncapi_compat project can be used to write a C extension supporting a wide range of Python versions with a single code base. It is made of the pythoncapi_compat.h header file and the upgrade_pythoncapi.py script." Documentation: https://pythoncapi-compat.readthedocs.io/en/latest/ GitHub: https://github.com/pythoncapi/pythoncapi_compat In the past, I managed to move my personal pyperf project under the PSF organization. Now other core developers are contributing and using it. It's better than having it as a personal project. pythoncapi_compat respects the PSF requirements to move a project in the GitHub PSF organization: contributors are required to respect the PSF Code of Conduct and the project has 3 maintainers (Dong-hee Na, Erlend E. AAsland and me). --- Some context. Incompatible C API changes in Python 3.10 and 3.11 require adding compatibility code to write C code compatible with old and new Python versions. For example, What's New in Python 3.10 suggests adding the following code to your project to get the Py_SET_REFCNT() function on Python 3.9 and older: #if PY_VERSION_HEX < 0x030900A4 # define Py_SET_REFCNT(obj, refcnt) ((Py_REFCNT(obj) = (refcnt)), (void)0) #endif Python 3.11 requires even more compatibility functions (7) and the implementation of these functions take more lines of code: * Py_SET_TYPE() * Py_SET_SIZE() * PyFrame_GetCode() * PyFrame_GetBack() * PyThreadState_GetFrame() * PyThreadState_EnterTracing() * PyThreadState_LeaveTracing() I added a reference to the pythoncapi_compat project to What's New in Python 3.11: "Or use the pythoncapi_compat project to get these functions on old Python functions." https://docs.python.org/dev/whatsnew/3.11.html#c-api-changes The project provides an upgrade_pythoncapi.py script to automatically add Python 3.10 and 3.11 support without losing support with Python 2.7. Just run the script, maybe copy pythoncapi_compat.h, and you're done! :-) It also provides a "pythoncapi_compat.h" header file which contains all the required compatibility code. Just copy the header file and use >#include "pythoncapi_compat.h"< in your project. You don't have to maintain this compatibilty code yourself anymore (pythoncapi_compat.h is made of 400 lines of C code). pythoncapi_compat supports Python (2.7 to 3.11), PyPy (2.7, 3.6, 3.7) and C++. It is distributed under the MIT license. My concern is that currently the project lives in my https://github.com/pythoncapi organization which is not really a sustainable option for the future. --- The pythoncapi_compat project is used by more and more Python projects, like Mercurial or mypy: https://pythoncapi-compat.readthedocs.io/en/latest/users.html I already made a similar request in June 2021 (move the project under the PSF organization): https://mail.python.org/archives/list/python-dev@python.org/thread/KHDZGCNOYEDUTSPAATUDP55ZSSQM5RRC/ What changed since June 2021? Python 3.11 now requires way more compatibility code than Python 3.10 does: 7 functions instead of 1. On Discord, I was asked to ask the Steering Council. I asked the SC who asked me to ask on python-dev, so here I am :-) Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XI23GU54UC2Y6UBMPY6US4IK4PSUXXPI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Request for feedback: pathlib.AbstractPath prototype
On 2/10/22 1:45 PM, Brett Cannon wrote: > Protocols would let folks rely on a common Path object API w/o having to require the object > come from pathlib itself or explicitly subclass something (which I admit would be rare, but > there's no reason to artificially constrain this either). Now maybe this is too broad of an > API for people to care, but since protocols are also ABCs it doesn't inherently make things > worse, either. So I would say subclassing Protocol makes sense while still using > abc.abstractmethod for methods people must implement. Brett, when you say Protocol are you referring to static typing? In your earlier email I thought you were referring to building blocks such as _fs_path, or the __iter__ and __next__ protocols. -- ~Ethan~ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HG6IF2YXGKXLYNZ7NNMZHU3UEQSFEHJ4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Request for feedback: pathlib.AbstractPath prototype
On Wed, Feb 9, 2022 at 11:59 AM Barney Gale wrote: > Penny for your thoughts on those questions, Brett? Protocols are new to > me. I see importlib.abc.Traversable is a Protocol, and I'm giving PEP 544 a > read now. > Protocols would let folks rely on a common Path object API w/o having to require the object come from pathlib itself or explicitly subclass something (which I admit would be rare, but there's no reason to artificially constrain this either). Now maybe this is too broad of an API for people to care, but since protocols are also ABCs it doesn't inherently make things worse, either. So I would say subclassing Protocol makes sense while still using abc.abstractmethod for methods people must implement. > > I reckon only stat(), open() and iterdir() would be essential for users to > implement. > Seems like a reasonable set to have to implement. -Brett > > On Wed, 9 Feb 2022 at 19:02, Brett Cannon wrote: > >> One thing to discuss (and which has been brought up on the PR), is >> whether this should be an ABC to force people to explicitly raise >> `NotImplementedError`? >> >> The next question is whether any of this should be a (very wide) protocol >> instead of an ABC? >> >> On Wed, Feb 9, 2022 at 7:05 AM Barney Gale wrote: >> >>> Over the last couple of years I've been tidying up the pathlib >>> internals, with a view towards adding an AbstractPath class to the >>> hierarchy. Users would be able to subclass AbstractPath to implement other >>> kinds of filesystems: s3, google cloud storage, pandas, ftp, git, zip and >>> tar files, etc. By implementing some abstract methods (stat(), iterdir(), >>> open(), etc) they'd benefit from a number of derived methods (is_dir(), >>> glob(), read_text(), etc). There's already a healthy ecosystem of PyPI >>> packages attempting this, but there's presently no officially-supported >>> route. >>> >>> I've now submitted a PR that adds an initial underscore-prefixed >>> implementation of this class: >>> https://github.com/python/cpython/pull/31085. The implementation is >>> simple: wherever Path calls functions in os, io, pwd, etc, AbstractPath >>> instead raises NotImplementedError. The Path class becomes an >>> implementation of AbstractPath. >>> >>> These methods directly raise NotImplementedError: cwd(), stat(), >>> iterdir(), readlink(), resolve(), expanduser(), owner(), group(), open(), >>> touch(), mkdir(), symlink_to(), hardlink_to(), rename(), replace(), >>> chmod(), unlink(), rmdir() >>> >>> These methods call through to the above methods: absolute(), lstat(), >>> exists(), is_dir(), is_file(), is_mount(), is_symlink(), is_block_device(), >>> is_char_device(), is_fifo(), is_socket(), home(), samefile(), scandir(), >>> glob(), rglob(), read_bytes(), read_text(), write_bytes(), write_text(), >>> lchmod(), link_to() >>> >>> Some methods aren't applicable to some kinds of filesystems, e.g. zip >>> files don't support symlinks, working directories or home directories. In >>> these cases I think it's reasonable to raise NotImplementedError. Indeed, >>> pathlib.Path methods already raise NotImplementedError when certain local >>> filesystem features aren't available (readlink(), group(), etc). >>> >>> If any readers of this list have previously tried to extend pathlib to >>> other domains, or are otherwise interested in pathlib development, please >>> could you let me know what you think of the proposed API? >>> >>> Many thanks >>> ___ >>> Python-Dev mailing list -- python-dev@python.org >>> To unsubscribe send an email to python-dev-le...@python.org >>> https://mail.python.org/mailman3/lists/python-dev.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-dev@python.org/message/JB7QGDNI2CNXFX7LQQ2X2WPOZ7DWVNQL/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CVU3MHGFLAIUMTCHMZBPLFJBW3MLMNAG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How to build cpython for x86_64 target on M1 Mac?
Thanks, Ned. I'm glad I didn't just miss it. I have built a universal binary, and it looks straightforward to extra the separate ARCHs out with lipo. Thanks for the response. Jeff ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TMO4YNX475CVQP4EXK2X4RHFZ3RPHMC2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How to build cpython for x86_64 target on M1 Mac?
On Feb 10, 2022, at 14:26, jeff@autodesk.com wrote: > We need to be able to produce both arm64 and x86_64 builds of cpython on a > single machine. I can see that, by default, it chooses the target > architecture based on the current running architecture. The docs say that we > can target one platform from the other, but it's not specific about how to do > that? > > I figure this is some sort of 'configure' option -- maybe "--build"? -- but > so far nothing I've tried has worked. > > Can someone help point us in the right direction? What's the magic to specify > one target or the other? We don't currently support cross-building a single macOS architecture on another, i.e. building an x86_64 only build on an Apple Silicon Mac. However, we do support building a multi-architecture build on either on current releases of Python 3.9 and 3.10, with what is known as a universal or fat build. In this case, you'd want to add these two options: ./configure --enable-universalsdk --with-universal-archs=universal2 ... For this to succeed, you will also need to provide universal builds of any of the third-party libraries used by standard modules you need and that are not otherwise provided by macOS, the most common being OpenSSL and XZ (and perhaps newer versions of SQLite and Tcl/Tk). For the most part, those are straightforward to build yourself but also may be available through some third-party distributors like MacPorts and possibly HomeBrew. The Mac/README.rst file in repo has some more details about universal builds and there may be some useful hints at building the third-party libraries in Mac/BuildScript/build-installer.py although it is not intended to be used for this purpose. The resulting universal2 build can be used as is on either Intel-64 or Apple Silicon Macs but, if you truly need to have only Intel binaries you should be able to use the macOS lipo command to separate out the desired architecture from the built executables and shared libraries into other files. If you want to run t hese binaries on a range of macOS versions, you can also set the MACOSX_DEPLOYMENT_TARGET environment variable when building the libs and Python; we currently test and support back to 10.9. -- Ned Deily n...@python.org -- [] ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/THJKS6X7RHB7WO4XRVDM5ABI2AKUXJG3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] How to build cpython for x86_64 target on M1 Mac?
Hi - Newbie post here... We need to be able to produce both arm64 and x86_64 builds of cpython on a single machine. I can see that, by default, it chooses the target architecture based on the current running architecture. The docs say that we can target one platform from the other, but it's not specific about how to do that? I figure this is some sort of 'configure' option -- maybe "--build"? -- but so far nothing I've tried has worked. Can someone help point us in the right direction? What's the magic to specify one target or the other? Thank you! Jeff ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OLCWTIDCFO4HNQHXZFIQ4F6R7ZFSDY5E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 677 (Callable Type Syntax): Rejection notice.
On Thu, Feb 10, 2022 at 3:32 AM Shantanu Jain wrote: > Hello! > > > Thanks for the decision, the points raised mostly make sense to me. > However, I find myself and a few others are a little confused by point 2. I > can read it as saying the following perhaps slightly contradictory things: > > > "It's good that PEP 677 is conservative and sticks to things Callable can > do" > > "PEP 677 isn't necessary, since Callable can do everything currently > proposed" > > "PEP 677 could be a slippery slope for further syntax expansions that can > do things Callable cannot" > > > Would the case for new syntax have been stronger if it was proposing > something Callable couldn't do? E.g., is the concern something like "The > cost of new syntax is best paid by expanding the realm of what is > expressible. While we see how PEP 677 could lead to such expansion in the > future, the merits of future expansion are currently uncertain and the > current proposal is too costly discussed on its own merits"? > > > Or is the concern forward compatibility in the eventuality of further > syntax expansions? PEP 677 did discuss "extended syntax", which the > proposed syntax would be forward compatible with. > https://www.python.org/dev/peps/pep-0677/#extended-syntax-supporting-named-and-optional-arguments > > > Or something else entirely! Would appreciate any clarity :-) > At least we're as consistent as the zen of python itself? ;) We collectively struggled with how to word that one. Sorry if it is confusing. >From my individual perspective, agreeing with point 2 came down to a belief that if we accepted the simple syntax, a use would arise in the future where it'd be desirable to gain something more complicated. Even if only "3%" of callable annotations use those *today*. So we expected to wind up in future releases with a proposal to extend the 677 callable syntax to support parsing a more complete def. Good design in that it appears technically feasible, but perceived bad in that the decision then would be to go forward into an even more complicated, even harder to parse, world or to back off and go another direction entirely which would leave 677 callable shorthand as a partial expressive dead end. Think of it as *attempting* not to accumulate too many ways to do the same thing in the long term. If the 677 proposed syntax had been expanded to include more def features instead of being conservative I think it would've been an even easier rejection for many of us as that would've triggered more "point 4" feelings of visual clutter and cognitive load. -gps > > Thank you! > > On Wed, 9 Feb 2022 at 19:45, Gregory P. Smith wrote: > >> Thank you PEP authors for producing a very well written and laid out PEP. >> That made it easy to understand the proposal, rationale, and alternatives >> considered. We agree that the existing typing.Callable syntax, while >> capable, is less than ideal for humans. That said, we deliberated last week >> and ultimately decided to reject PEP-677 Callable Type Syntax. >> >> Why? Several considerations led us here: >> >> 1. We feel we need to be cautious when introducing new syntax. Our new >> parser presents understandably exciting opportunities but we don’t want its >> existence to mean we add new syntax easily. A feature for use only in a >> fraction of type annotations, not something every Python user uses, did not >> feel like a strong enough reason to justify the complexity needed to parse >> this new syntax and provide meaningful error messages. Not only code >> complexity, humans are also parsers that must look forwards and backwards. >> >> 2. While the current Callable[x, y] syntax is not loved, it does work. >> This PEP isn’t enabling authors to express anything they cannot already. >> The PEP explicitly chose be conservative and not allow more syntax to >> express features going beyond what Callable supports. We applaud that >> decision, starting simple is good. But we can imagine a future where the >> syntax would desire to be expanded upon. >> >> 3. In line with past SC guidance, we acknowledge challenges when syntax >> desires do not align between typing and Python itself. Each time we add >> syntax solely for typing it shifts us further in the direction of typing >> being its own mini-language so we aim to tread lightly in what gets added >> here. Adopting PEP 677 would lock us into its syntax forever, potentially >> preventing other future syntax directions. >> >> 4. We did not like the visual and cognitive consequence of multiple `->` >> tokens in a def. Especially when code is not formatted nicely. Though we >> admit the correlation between Python typing and formatter users is high. >> >> ## Recommendation for future syntax enhancements: >> >> When proposing a syntax change, low complexity is better. While not >> always possible, it’s ideal if it could still be described using our old >> <=3.8 parser. It is important to have in mind that adding syntax that only >> our modern PEG
[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)
Petr Viktorin schrieb am 10.02.22 um 11:22: So, should there be a mechanism to set source/lineno/position on tracebacks/exceptions, rather than always requiring a frame for it? There's "_PyTraceback_Add()" currently, but it's incomplete in terms of what Cython would need. As it stands, Cython could make use of a function that accepted - string object arguments for filename and function name - (optionally) a 'globals' dict (or a reference to the current module) - (optionally) a 'locals' mapping - (optionally) a code object - a C integer source line - a C integer position, probably start and end lines and columns to add a traceback level to the current exception. I'm not sure about the code object since that's a rather heavy thing, but given that Cython needs to create code objects in order for its functions to be introspectible, that seems like a worthwhile option to have. However, with the recent frame stack refactoring and frame object now being lazily created, according to https://bugs.python.org/issue44032 https://bugs.python.org/issue44590 I guess Cython should rather integrate with the new stack frame infrastructure in general. That shifts the requirements a bit. An API function like the above would then still be helpful for the reduced API compile mode, I guess. But as soon as Cython uses InterpreterFrame structs internally, it would no longer be helpful for the fast mode. InterpreterFrame object are based on byte code instructions again, which brings us back to co_positions. Stefan ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YSP36JL5SRSPEG4X67G5RMWUWLVXSDC5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 677 (Callable Type Syntax): Rejection notice.
Hello! Thanks for the decision, the points raised mostly make sense to me. However, I find myself and a few others are a little confused by point 2. I can read it as saying the following perhaps slightly contradictory things: "It's good that PEP 677 is conservative and sticks to things Callable can do" "PEP 677 isn't necessary, since Callable can do everything currently proposed" "PEP 677 could be a slippery slope for further syntax expansions that can do things Callable cannot" Would the case for new syntax have been stronger if it was proposing something Callable couldn't do? E.g., is the concern something like "The cost of new syntax is best paid by expanding the realm of what is expressible. While we see how PEP 677 could lead to such expansion in the future, the merits of future expansion are currently uncertain and the current proposal is too costly discussed on its own merits"? Or is the concern forward compatibility in the eventuality of further syntax expansions? PEP 677 did discuss "extended syntax", which the proposed syntax would be forward compatible with. https://www.python.org/dev/peps/pep-0677/#extended-syntax-supporting-named-and-optional-arguments Or something else entirely! Would appreciate any clarity :-) Thank you! On Wed, 9 Feb 2022 at 19:45, Gregory P. Smith wrote: > Thank you PEP authors for producing a very well written and laid out PEP. > That made it easy to understand the proposal, rationale, and alternatives > considered. We agree that the existing typing.Callable syntax, while > capable, is less than ideal for humans. That said, we deliberated last week > and ultimately decided to reject PEP-677 Callable Type Syntax. > > Why? Several considerations led us here: > > 1. We feel we need to be cautious when introducing new syntax. Our new > parser presents understandably exciting opportunities but we don’t want its > existence to mean we add new syntax easily. A feature for use only in a > fraction of type annotations, not something every Python user uses, did not > feel like a strong enough reason to justify the complexity needed to parse > this new syntax and provide meaningful error messages. Not only code > complexity, humans are also parsers that must look forwards and backwards. > > 2. While the current Callable[x, y] syntax is not loved, it does work. > This PEP isn’t enabling authors to express anything they cannot already. > The PEP explicitly chose be conservative and not allow more syntax to > express features going beyond what Callable supports. We applaud that > decision, starting simple is good. But we can imagine a future where the > syntax would desire to be expanded upon. > > 3. In line with past SC guidance, we acknowledge challenges when syntax > desires do not align between typing and Python itself. Each time we add > syntax solely for typing it shifts us further in the direction of typing > being its own mini-language so we aim to tread lightly in what gets added > here. Adopting PEP 677 would lock us into its syntax forever, potentially > preventing other future syntax directions. > > 4. We did not like the visual and cognitive consequence of multiple `->` > tokens in a def. Especially when code is not formatted nicely. Though we > admit the correlation between Python typing and formatter users is high. > > ## Recommendation for future syntax enhancements: > > When proposing a syntax change, low complexity is better. While not always > possible, it’s ideal if it could still be described using our old <=3.8 > parser. It is important to have in mind that adding syntax that only our > modern PEG parser can handle could lead to greater cognitive load and > external tooling implementation costs. > > This should not be read as a ban on PEG-only syntax, we just think it > should be used for broadly applicable features or else be relatively > unintrusive. > > Thanks, > The 3.11 Python Steering Council > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/NHCLHCU2XCWTBGF732WESMN42YYVKOXB/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MCCF22SMV5VZGGQLYVIV7KECT7KH3F2W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11
On Thu, Feb 10, 2022 at 10:28 AM Petr Viktorin wrote: > Ah, I've also wanted anonymous unions in the past! > There's a little problem in that they're not valid in C++, so we can't > have them in public headers. > > We'll need to mention C++ if we update the standard. IMO we only have to care about C++ in Include/ header files: make sure that "Python.h" is usable in C++. Outside this directory, we don't have to care about C++ compatibility. Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AYRNS2PGB6MOUPEY5IJXFMGEVW4T5L6E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API
On Thu, Feb 10, 2022 at 10:58 AM Petr Viktorin wrote:
>
> On 09. 02. 22 21:41, Gregory P. Smith wrote:
> >
> > On Wed, Feb 9, 2022 at 8:54 AM Victor Stinner > Perhaps use a hybrid approach
> > when feasible similar to:
> >#define PyUnicode_CHECK_INTERNED(op)
> > _PyUnicode_CHECK_INTERNED((PyASCIIObject *)(op))
> >
> > That should make it safe. And I believe you do mention this technique
> > as something to be used in the PEP.
It is exactly what the PEP proposes in the Specification section to
avoid adding new compiler warnings:
https://www.python.org/dev/peps/pep-0670/#cast-to-pyobject
Macro which is only there to add the cast, to avoid adding a compiler warning:
#define Py_TYPE(ob) _Py_TYPE(_PyObject_CAST_CONST(ob))
Static inline function:
static inline PyTypeObject* _Py_TYPE(const PyObject *ob) { ... }
Use the existing macro to do the cast:
#define _PyObject_CAST_CONST(op) ((const PyObject*)(op))
> That technique is mentioned in the PEP, but it looks like we just found
> a better way to do it than what the PEP suggests: the macro and the
> function can have the same name
Well, when I started to convert macros to static inline functions, I
chose to have a different name to help *me* identifying what is what:
"Pyxxx" is the macro and "_Pyxxx" is the static inline functions. It
looked convenient at the beginning. It looked like a good idea...
Then I started to see "_Pyxxx" functions in debuggers and profilers.
It's not convenient since in C extensions, the public "Pyxxx" (macro)
name is used, same in old Python versions. Having a different name can
cause practical issues: Lib/test/test_gdb.py and
Tools/gdb/libpython.py have to search for multiple names for the same
function in this case (to support old and new Python versions).
The new private/secret "_Pyxxx" name is unexpected, so I proposed to
abandon this bad habit and use the same name for the macro and for the
static inline function.
> Are there any downsides to that?
There is no downside. Macros are only for the C preprocessor, they are
gone at the ABI level. The macro is just there to add a cast to avoid
adding new compiler warnings (since the old macro already did that).
Moreover, PEP 670 proposes to replace some static inline functions
with regular functions to make them available for extension modules
which cannot use static inline functions, like the vim text editor
(written in C) which only loads symbols from libpython. Having a
consistent name for macros, static inline functions and functions
should help for that.
--
There is one exception: when a function as exposed as static inline
function *and* a regular function. _Py_Dealloc() and Py_NewRef() for
example.
Py_NewRef() is defined as a macro which calls _Py_NewRef() static
inline function (for best performance), and there is a regular
Py_NewRef() regular function (exposed in the ABI): the regular
function and the static inline functions cannot have the same name:
PyAPI_FUNC(PyObject*) Py_NewRef(PyObject *obj);
static inline PyObject* _Py_NewRef(PyObject *obj)
{ Py_INCREF(obj); return obj; }
#define Py_NewRef(obj) _Py_NewRef(_PyObject_CAST(obj))
Names:
* Macro: Py_NewRef
* Regular function: Py_NewRef
* Static inline function: _Py_NewRef (different name)
If _Py_NewRef() is renamed to Py_NewRef(), the compiler fails with:
./Include/object.h:597:25: error: static declaration of 'Py_NewRef'
follows non-static declaration
Another example is _Py_Dealloc() declared as a macro+static inline
function (best performance) *and* a regular function (expose it in the
ABI): the static inline function has a different name.
IMO the most important name is the regular function name since it's
the one which lands in libpython ABI. Static inline functions are
*not* part of the libpython ABI, they are either inlined or copied as
regular functions (depending on what the compiler prefers) in each
extension module using it.
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/XOLV4BVJDZH3RJRLV3UKDYKA664IANZO/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)
On 09. 02. 22 20:04, Stefan Behnel wrote: Guido van Rossum schrieb am 09.02.22 um 19:36: On Wed, Feb 9, 2022 at 9:41 AM Pablo Galindo Salgado wrote: On Wed, 9 Feb 2022 at 17:38, Stefan Behnel wrote: Pablo Galindo Salgado schrieb am 09.02.22 um 17:40: Should there be a getter/setter for co_positions? We consider the representation of co_postions private Yes, and that's the issue. I can only say that currently, I am not confident to expose such an API, at least for co_positions, as the internal implementation is very likely to heavily change and we want to have the possibility of changing it between patch versions if required (to address bugs and other things like that). > > It might require a detailed API design proposal coming from outside > CPython > (e.g. from Cython) to get this to change. I imagine for co_positions in > particular this would have to use a "builder" pattern. > > I am unclear on how this would work though, given that Cython generates C > code, not CPython bytecode. How would the synthesized co_positions be > used? > Would Cython just generate a co_positions fragment at the moment an > exception is raised, pointing at the .pyx file from which the code was > generated? So, what we currently do is to update the line number (which IIRC is really the start line number of the current function) on the current frame when an exception is raised, and the byte code offset to 0. That's a hack but shows the correct code line in the traceback. Probably conflicts with pdb, but there are still other issues with that anyway. So, should there be a mechanism to set source/lineno/position on tracebacks/exceptions, rather than always requiring a frame for it? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6AVY3W7PLUFAYQE26JEW3F5IUY2NBA6N/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API
On 09. 02. 22 21:41, Gregory P. Smith wrote: On Wed, Feb 9, 2022 at 8:54 AM Victor Stinner [...] Two differing examples from that PR 31221: Hold off as unsafe for now: Macros that do things like (PyWhateverObject*)(op) such as PyUnicode_CHECK_INTERNED(op) should not have the casting part of macros replaced yet. Doing so could result in a warning or failure at -Werror compile time if someone was not using a PyObject*. Code is /supposed/ to, but the cast means anyone could've used PyUnicodeObject or whatnot itself. Perhaps use a hybrid approach when feasible similar to: #define PyUnicode_CHECK_INTERNED(op) _PyUnicode_CHECK_INTERNED((PyASCIIObject *)(op)) That should make it safe. And I believe you do mention this technique as something to be used in the PEP. That technique is mentioned in the PEP, but it looks like we just found a better way to do it than what the PEP suggests: the macro and the function can have the same namea Are there any downsides to that? We don't know of any so far. Is it a widely used trick in some other codebases? Should we try to find to people who use it regularly and ask what to do and what to watch out for? If we just approved the PEP, a lot of the macros would probably get converted to what the PEP suggests. With a bit more time and discussion, we can arrive at a better solution. This is a large change across the whole codebase. It won't be easy to fix it up later. It won't be easy to revert it if things go wrong. IMO this is a case where "never is better than *right* now". But it doesn't need to be "never". ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RSNDG7NTQZYPR6ZDHDNZPY7JONXBLVUF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11
On 10. 02. 22 0:30, Inada Naoki wrote: On Thu, Feb 10, 2022 at 3:49 AM Brett Cannon wrote: On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin wrote: On 09. 02. 22 4:39, h.vetin...@gmx.com wrote: That's an interesting idea -- what's keeping us from C11? No one asking before, probably because we have been trying to get to C99 for so long. In other words: the main thing keeping us from C99 is MSVC support, and since that compiler apparently skipped C99, should we skip it as well? If we think "C11 without optional features" is widely supported then I think that's a fine target to have. For anyone not sure what's optional in C11, I found https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_features . Other than atomics being discussed on Discord for mimalloc, leaving those things out seem reasonable to me. I like it. I want to use anonymous union. It makes complex structure like PyDictKeysObject simple a little. I confirmed that XLC supports it. https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-unions#strct__anonstruct Ah, I've also wanted anonymous unions in the past! There's a little problem in that they're not valid in C++, so we can't have them in public headers. We'll need to mention C++ if we update the standard. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XLRK7P3Q4ANNC4ZEIPIHWJHFKFCWWMRD/ Code of Conduct: http://python.org/psf/codeofconduct/
