[Python-Dev] Cycles in the __context__ chain
It is possible to create a loop by setting the __context__ attribute of
the raised exception, either explicitly, or implicitly, using "raise ...
from ...". Creating a loop can leads to the hanging in infinite loop or
crash due to stack overflow. Several years ago we did have issues
related to ExitStack and contextmanager() [1] [2], and recently we have
found other issue related to asynccontextmanager() [3]. There may be
other issues in the stdlib or third-party libraries.
To solve all such issues we can prevent creating the loop in the
__context__ setter. Unfortunately there is no way to return an error in
PyException_SetContext(), so it is always success. So we should break a
loop at some point. There was proposed several options. If we have a chain
A -> B -> C -> D -> E -> NULL
after raising C in the context of A ("raise C from A") or setting
C.__context__ = A we will get a chain:
0. C -> A -> B -> C -> A -> B -> ...
A loop. This is the current behavior.
1. C -> D -> E -> NULL
The operation is no-op. We lose information about exceptions preceding
C: A and B.
2. C -> NULL
Lose all context.
3. C -> A -> B -> D -> E -> NULL
C is moved at the front of the chain. No exception is lost, but their
order is changed.
4. C -> A -> B -> NULL
Remove C from the chain and rewrite its context. We lose information
about the old context of C: D and E.
5. C -> A -> B -> C' -> D -> E -> NULL
Create a copy of C (how?) and replace C with its copy in the chain. But
what to do with other references to C?
There may be other options.
We need to solve this issue, otherwise we will continue to stick with
other bugs related to the exceptions loop.
[1] https://bugs.python.org/issue25779
[2] https://bugs.python.org/issue25786
[3] https://bugs.python.org/issue40696
[4] https://bugs.python.org/issue25782
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/MHH2BOG2BIK7MLGKZP7Q5S6MQDCJPF3S/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] How to specify optional support of arguments
In Python 3.3 a number of functions in the os module got the support of new options, like dir_fd or follow_symlinks. I well remember this because the initial reason of my contribution to CPython was related to this feature. The support of new options was platform dependent. They were supported only on some Posix platforms, but not on Windows. Sets like supports_dir_fd and supports_follow_symlinks was added in the os module to specify which functions support which options. If say os.stat is contained in os.supports_dir_fd, then os.stat() supports keyword argument dir_fd. Now I want to add the support of dir_fd to glob.glob() and some other functions. It will be platform-dependent, it requires os.open() and os.stat() supporting dir_fd and os.scandir supporting the file descriptor argument. How can I specify that glob.glob() supports dir_fd? 1. Add glob.glob in os.supports_dir_fd? It looks strange that importing the glob module modifies the value of the os module. 2. Introduce the glob.supports_dir_fd set and add glob.glob to it? We will need to add sets supports_dir_fd in all modules which contain functions which can support dir_fd. But what if the function is defined in one module, by normally is imported from other module? Should we merge sets supports_dir_fd from different modules? Also, the common problem with two former options is that when we wrap the function, we need to add it to the corresponding set. 3. Or set the attribute glob.glob.supports_dir_fd = True or glob.glob.__supports_dir_fd__ = True (and add similar attributes to all os functions which support)? What are your thoughts? ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/GCPDYSPB4656Q7TAMDLBFED6NQJCNLIP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Proposed release schedule for 3.5.9
Gosh how time flies. 3.5 is nearing the end of its security maintenance period, and as such the patches have slowed down. But we got a couple recently--all having to do with web technologies--so it's probably time to make another release. I propose to release rc1 in two weeks, and final two weeks after that. Specifically: Sunday Jun 28 - 3.5.9rc1 Sunday Jul 12 - 3.5.9 final Of course, since we're in the security-fixes-only period, these will all be source-code only releases. Let me know if this is too soon for you. My suspicion is it's no problem. It's mainly one developer filing security patches for 3.5 anyway (hi Victor!) and I'll contact him directly to make sure. 3.5 will be 5 years old this August, at which point we'll formally end 3.5 support. I expect there'll be only one more release after that, 3.5.10, and that'll be it. At which point, at last! you'll be rid of me as a release manager! Surely on that day the programming world will breathe a collective sigh of relief. Best wishes, //arry/ ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/OPNTLCQL7TMF67UBW7LI7PSUTRB36NRW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Removal of _Py_ForgetReference from public header in 3.9 issue
Please excuse if this is the wrong mailing list. I couldn't find one for module maintainers. I maintain an open source Python module in C. I'm trying to verify for the first time that the module still works with cpython 3.9. This module does *not* use the "limited" C API. In building my module against 3.9b3, I'm getting a missing declaration warning on _Py_ForgetReference. My module builds and passes test fine, this is just a compiler warning issue. The change that caused this was made in: commit f58bd7c1693fe041f7296a5778d0a11287895648 Author: Victor Stinner Date: Wed Feb 5 13:12:19 2020 +0100 bpo-39542: Make PyObject_INIT() opaque in limited C API (GH-18363) ... I definitely need the _Py_ForgetReference call for a particularly hairy error condition (https://github.com/jnwatson/py-lmdb/blob/master/lmdb/cpython.c#L888 if you're curious). In fact, my tests will seg fault if I don't have that call and trace refs is enabled. Should I put an #ifdef Py_TRACE_REFS around the call? Ignore it? What do you think is the proper resolution? Nic Watson ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/CQYVR7TZZITURBZKVWIEOBGF343GI52W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How to specify optional support of arguments
(see an answer to the OP's specific question below) I don't quite get the point for .supports_stuff. It seems well-intentioned but misguided. 1. The documentation clearly says that it's supported depending on OS flavor -- so if I want to know if I can supply it, I need to rather check `os.name`. Those members are thus redundant. If the distinction is finer than os.name then I'd need some other, case-specific check depending on what the distinction is; alternatively, I could check the function's signature in its metadata if the support is reflected in it. 2. Support of a parameter is a characteristic of a function rather than a module. So information like this doesn't belong at module level. Moreover, object references can be moved around at will and they are all equal, there's no such thing as "one true reference" (you can get the object's lexical scope but there's no guarantee that it corresponds to the public interface). This is the structural conflict that Sergey ran into when he suddenly discovered that the functions and the flag are two completely unrelated objects that can be in completely different places. So any such flags, if they have the right to exist at all, should be attached to function objects. On 14.06.2020 20:36, Serhiy Storchaka wrote: In Python 3.3 a number of functions in the os module got the support of new options, like dir_fd or follow_symlinks. I well remember this because the initial reason of my contribution to CPython was related to this feature. The support of new options was platform dependent. They were supported only on some Posix platforms, but not on Windows. Sets like supports_dir_fd and supports_follow_symlinks was added in the os module to specify which functions support which options. If say os.stat is contained in os.supports_dir_fd, then os.stat() supports keyword argument dir_fd. Now I want to add the support of dir_fd to glob.glob() and some other functions. It will be platform-dependent, it requires os.open() and os.stat() supporting dir_fd and os.scandir supporting the file descriptor argument. How can I specify that glob.glob() supports dir_fd? 1. Add glob.glob in os.supports_dir_fd? It looks strange that importing the glob module modifies the value of the os module. 2. Introduce the glob.supports_dir_fd set and add glob.glob to it? We will need to add sets supports_dir_fd in all modules which contain functions which can support dir_fd. But what if the function is defined in one module, by normally is imported from other module? Should we merge sets supports_dir_fd from different modules? This would be the way consistent with the current design. If the public interface is to use the function from another module, the public interface module should have a parameter like this; whether the private module also has one and whether they are different objects is implementation detail. Also, the common problem with two former options is that when we wrap the function, we need to add it to the corresponding set. 3. Or set the attribute glob.glob.supports_dir_fd = True or glob.glob.__supports_dir_fd__ = True (and add similar attributes to all os functions which support)? What are your thoughts? ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/GCPDYSPB4656Q7TAMDLBFED6NQJCNLIP/ Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/F4N5RY3VFSFEYT57RZMPA74J67XAGU45/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposed release schedule for 3.5.9
3.5.9 was already released back in November. Yes, it's (almost) the same code as 3.5.8, but its version number (as reported by `python -V` etc.) is still 3.5.9. Releasing another 3.5.9 would just lead to confusion at best. I feel that the July release should be numbered 3.5.10 instead. -- John Wodder ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/7Z6HBMUYGTB55QEASAOVML4VIDI3TKVS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How to specify optional support of arguments
Instead of a bunch of ad-hoc mechanisms for finding out about platform-dependent arguments, maybe there should be a function in the inspect module for testing whether a function has a given argument. Then you could say something like if inspect.hasargument(glob.glob, 'dir_fd'): ... -- Greg ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/EPXHZ54KPWUP6MAEDUEPXA2VK4SBH7EC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When can we remove wchar_t* cache from string?
On Sat, Jun 13, 2020 at 8:20 PM Inada Naoki wrote: > > 2020年6月13日(土) 20:12 Kyle Stanley : >> >> > Additionally, raise DeprecationWarning runtime when these APIs are used. >> >> So, just to clarify, current usage of these 7 unicode APIs does not emit any >> warnings and would only start doing so in 3.10? > > They have been deprecated in C already. Compiler emits warning. > > This additional proposal is adding runtime warning before removal. > I'm sorry, I was wrong. Py_DEPRECATED(3.3) is commented out for some APIs. So Python 3.8 doesn't show warning for them. I want to uncomment them in Python 3.9. https://github.com/python/cpython/pull/20878 As far as I grepped, most of PyPI packages use deprecated APIs because Cython generates it. Updating Cython will fix them. Some of them are straightforward and I have created an issue or sent pull request already. A few projects, pyScss and Genshi are not straightforward. But it is not too hard and I will help them. I still think 2 years are enough to removal. Regards, ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/4GVB4LK63VRCEEGW7RYRC7TAZKTWSLPH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How to specify optional support of arguments
14.06.20 23:45, Ivan Pozdeev via Python-Dev пише: 1. The documentation clearly says that it's supported depending on OS flavor -- so if I want to know if I can supply it, I need to rather check `os.name`. Those members are thus redundant. If the distinction is finer than os.name then I'd need some other, case-specific check depending on what the distinction is; alternatively, I could check the function's signature in its metadata if the support is reflected in it. Yes, it is finer than os.name. It can depend on the kernel or libc version (and we do not know which versions are required on every platform), and there are a lot of platforms besides the main three. The user should not be expert in all platforms on which his program potentially can be ran. The function's signature is the same on all platforms. Just on some platforms only default value can be passed (None for dir_fd) or only specific types of argument is accepted (path-like, but not int). ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/YWYTZRLQK4ZHRFX3G3MJDGN6H4BJQCKN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How to specify optional support of arguments
On 15.06.2020 8:45, Serhiy Storchaka wrote: 14.06.20 23:45, Ivan Pozdeev via Python-Dev пише: 1. The documentation clearly says that it's supported depending on OS flavor -- so if I want to know if I can supply it, I need to rather check `os.name`. Those members are thus redundant. If the distinction is finer than os.name then I'd need some other, case-specific check depending on what the distinction is; alternatively, I could check the function's signature in its metadata if the support is reflected in it. Yes, it is finer than os.name. It can depend on the kernel or libc version (and we do not know which versions are required on every platform), and there are a lot of platforms besides the main three. The user should not be expert in all platforms on which his program potentially can be ran. The function's signature is the same on all platforms. Just on some platforms only default value can be passed (None for dir_fd) or only specific types of argument is accepted (path-like, but not int). Then a single boolean flag is clearly not enough. Compare: in https://docs.python.org/3/library/resource.html , the set of present RLIMIT_* members shows which rlimits are available in the specific OS. So I guess you want some similar pointers that would show which relevant #define constants (or other C-level entities that govern the availability) were present at the time of compilation. If availability is rather governed by a runtime than compile-time check, then something to perform the same check should be introduced; the distinction from .supports_stuff is it would be named after the check and completely decoupled from any functions that might be affected by the check. ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/YWYTZRLQK4ZHRFX3G3MJDGN6H4BJQCKN/ Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/BJ7HFKNQAO3C44XQ75AEYCRTG5GMYTU5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How to specify optional support of arguments
15.06.20 04:06, Greg Ewing пише: Instead of a bunch of ad-hoc mechanisms for finding out about platform-dependent arguments, maybe there should be a function in the inspect module for testing whether a function has a given argument. Then you could say something like if inspect.hasargument(glob.glob, 'dir_fd'): ... How can the inspect module know? The function's signature is the same. Just not all values or types are accepted on all platforms. ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/HKQ6J7E7QQJHS5YWP4KDPLJLQFUBPPZ4/ Code of Conduct: http://python.org/psf/codeofconduct/
