[Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Victor Stinner
>> The very first question I asked was whether this would let us converge >> the ABIs, and the answer was "no". The answer is yes and it's my primary goal. See my first email: "This change makes the debug build ABI closer to the release build ABI". To be honest, I am now lost in this long thread

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Nathaniel Smith
On Fri, Apr 12, 2019 at 5:05 PM Steve Dower wrote: > > On 12Apr.2019 1643, Nathaniel Smith wrote: > > On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: > >> > >> On 10Apr2019 1917, Nathaniel Smith wrote: > > I don't know how many people use Py_TRACE_REFS, but if we can't find > > anyone on pytho

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Steve Dower
On 12Apr.2019 1643, Nathaniel Smith wrote: > On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: >> >> On 10Apr2019 1917, Nathaniel Smith wrote: > I don't know how many people use Py_TRACE_REFS, but if we can't find > anyone on python-dev who uses it then it must be pretty rare. If > dropping Py_TR

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Nathaniel Smith
On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: > > On 10Apr2019 1917, Nathaniel Smith wrote: > > It sounds like --with-pydebug has accumulated a big grab bag of > > unrelated features, mostly stuff that was useful at some point for > > some CPython dev trying to debug CPython itself? It's clea

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Victor Stinner
Le sam. 13 avr. 2019 à 00:38, Greg Ewing a écrit : > Rather than removing features altogether, maybe the debug > build could be split into a number of separate features > that can be enabled individually? I don't propose to *remove* a feature, but just to *disable* it *by default* (when Python is

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Greg Ewing
Victor Stinner wrote: I'm not sure of what you means by "objects placed at static memory": the double linked list of all Python objects is created at runtime. _ob_next and _ob_prev are initialized statically to NULL. The trick of allocating extra memory in front of the object would be harder to

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Greg Ewing
Victor Stinner wrote: Python is used on devices with low memory (ex: 256 MiB for the whole system). Allowing developers to use a debug build on such devices seem to be a legit rationale for such change. Rather than removing features altogether, maybe the debug build could be split into a number

Re: [Python-Dev] Removing PID check from signal handler

2019-04-12 Thread Ivan Pozdeev via Python-Dev
On 12.04.2019 21:05, Steve Dower wrote: On 12Apr.2019 0919, Jeroen Demeyer wrote: The signal handler (that receives signals from the OS) in Python starts with a check     if (getpid() == main_pid) Looking at the comments, the intent was to do a check for the main *thread* but this is checking

Re: [Python-Dev] Update PEP 394: Distributions can choose what does python command mean

2019-04-12 Thread Barry Warsaw
Thanks for the update. I have made one small suggestion on the PR for clarification, but otherwise the changes LGTM. -Barry > On Apr 12, 2019, at 07:53, Miro Hrončok wrote: > > Hello. > > Based on discussions in [1], Petr Viktorin and me have drafted a new update > [2] to the PEP 394 (The "

[Python-Dev] Summary of Python tracker Issues

2019-04-12 Thread Python tracker
ACTIVITY SUMMARY (2019-04-05 - 2019-04-12) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open7063 ( +7) closed 41307 (+76) total 48370 (+83) Open issues wi

Re: [Python-Dev] Removing PID check from signal handler

2019-04-12 Thread Steve Dower
On 12Apr.2019 0919, Jeroen Demeyer wrote: > The signal handler (that receives signals from the OS) in Python starts > with a check > >     if (getpid() == main_pid) > > Looking at the comments, the intent was to do a check for the main > *thread* but this is checking the *process* id. So this con

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-12 Thread Brett Cannon
On Fri, Apr 12, 2019 at 8:35 AM Serhiy Storchaka wrote: > 12.04.19 16:44, Inada Naoki пише: > > When creating many dicts with same keys, dict need to > > lookup internal hash table while inserting each keys. > > > > It is costful operation. If we can reuse existing keys of dict, > > we can skip

[Python-Dev] Removing PID check from signal handler

2019-04-12 Thread Jeroen Demeyer
The signal handler (that receives signals from the OS) in Python starts with a check if (getpid() == main_pid) Looking at the comments, the intent was to do a check for the main *thread* but this is checking the *process* id. So this condition is basically always true. Therefore, I sugges

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-12 Thread Inada Naoki
On Sat, Apr 13, 2019 at 12:38 AM Serhiy Storchaka wrote: > > It looks contrary to simplification made in Python 3 when we get rid of > some more efficient lists in favor of more general iterators. > Yes. This is API for special use case creates many dict having same keys, like csv.DictReader. It

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-12 Thread Inada Naoki
On Fri, Apr 12, 2019 at 11:31 PM Victor Stinner wrote: > > Nice optimization! I have questions on the proposed API. > > > with_values(self, iterable, /) > > Create a new dictionary with keys from this dict and values from > > iterable. > > > > When length of iterable is different from len

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-12 Thread Serhiy Storchaka
12.04.19 16:44, Inada Naoki пише: When creating many dicts with same keys, dict need to lookup internal hash table while inserting each keys. It is costful operation. If we can reuse existing keys of dict, we can skip this inserting cost. Additionally, we have "Key-Sharing Dictionary (PEP 412)

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Serhiy Storchaka
12.04.19 17:40, Guido van Rossum пише: So let's leave it off by default even in debug builds. But let's not delete the macros. Maybe switch it on (together with other disabled by default options) on some fast buildbot? ___ Python-Dev mailing list P

Re: [Python-Dev] Update PEP 394: Distributions can choose what does python command mean

2019-04-12 Thread Guido van Rossum
I think this is reasonable. Thanks for making the rationale clear! On Fri, Apr 12, 2019 at 8:02 AM Petr Viktorin wrote: > On 4/12/19 4:53 PM, Miro Hrončok wrote: > > Hello. > > > > Based on discussions in [1], Petr Viktorin and me have drafted a new > > update [2] to the PEP 394 (The "python" Co

Re: [Python-Dev] Update PEP 394: Distributions can choose what does python command mean

2019-04-12 Thread Petr Viktorin
On 4/12/19 4:53 PM, Miro Hrončok wrote: Hello. Based on discussions in [1], Petr Viktorin and me have drafted a new update [2] to the PEP 394 (The "python" Command on Unix-Like Systems). The update gives distributors the opportunity to decide where does the "python" command lead to, whether

[Python-Dev] Update PEP 394: Distributions can choose what does python command mean

2019-04-12 Thread Miro Hrončok
Hello. Based on discussions in [1], Petr Viktorin and me have drafted a new update [2] to the PEP 394 (The "python" Command on Unix-Like Systems). The update gives distributors the opportunity to decide where does the "python" command lead to, whether it is present etc. Please, see the PR [

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Guido van Rossum
On Fri, Apr 12, 2019 at 5:51 AM Stefan Behnel wrote: > Serhiy Storchaka schrieb am 11.04.19 um 17:30: > > If reducing the Python memory footprint is an argument for disabling > > Py_TRACE_REFS, it is a weak argument because there is larger overhead in > > the debug build. > > I think what Victor

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-12 Thread Victor Stinner
Nice optimization! I have questions on the proposed API. > with_values(self, iterable, /) > Create a new dictionary with keys from this dict and values from iterable. > > When length of iterable is different from len(self), ValueError is raised. > This method does not support dict subc

[Python-Dev] Proposal: dict.with_values(iterable)

2019-04-12 Thread Inada Naoki
Hi, all. I propose adding new method: dict.with_values(iterable) # Motivation Python is used to handle data. While dict is not efficient way to handle may records, it is still convenient way. When creating many dicts with same keys, dict need to lookup internal hash table while inserting each k

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Stefan Behnel
Serhiy Storchaka schrieb am 11.04.19 um 17:30: > If reducing the Python memory footprint is an argument for disabling > Py_TRACE_REFS, it is a weak argument because there is larger overhead in > the debug build. I think what Victor is argueing is rather that we have better ways to debug memory pro

Re: [Python-Dev] PEP-582 and multiple Python installations

2019-04-12 Thread Victor Stinner
Hi, Le mar. 2 avr. 2019 à 17:20, Calvin Spealman a écrit : > While the PEP does show the version number as part of the path to the actual > packages, implying support for multiple versions, this doesn't seem to be > spelled out in the actual text. Presumably __pypackages__/3.8/ might sit > bes

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Victor Stinner
Le ven. 12 avr. 2019 à 12:57, Victor Stinner a écrit : > I wrote https://bugs.python.org/issue36611 to remove the serialno field > of debug hooks on Python memory allocators: it reduces > the memory footprint by 5% (ex: 1.2 MiB on 33.0 MiB when running > test_asyncio). I measured the memory footp

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Victor Stinner
Le jeu. 11 avr. 2019 à 17:33, Serhiy Storchaka a écrit : > If reducing the Python memory footprint is an argument for disabling > Py_TRACE_REFS, it is a weak argument because there is larger overhead in > the debug build. The "serialno" field of debug memory allocators is documented as: "an excel