Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Inada Naoki
On Fri, May 24, 2019 at 3:49 AM Gregory P. Smith wrote: > > I suggest filing a bug to track this... > I created the issue: https://bugs.python.org/issue37029 -- Inada Naoki ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Greg Ewing
Steve Dower wrote: We need to make it more clear somehow that Python uses series.major.minor.micro versioning, not SemVer. We could also do with deciding what the "series" part is really supposed to be for. At the moment it seems to be "we change it when the major number gets up to 9, or we

Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done (was: PEP 594: Removing dead batteries from the standard library)

2019-05-23 Thread Barry Warsaw
On May 23, 2019, at 14:17, Brett Cannon wrote: > I'm personally viewing it as a first step in addressing the maintenance > burden we have with such a large stdlib. Christian started this work over a > year ago and I think it's worth seeing through. After that we should probably > have a

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Steven D'Aprano
On Thu, May 23, 2019 at 02:06:13PM -0700, Brett Cannon wrote: > On Wed, May 22, 2019 at 1:23 PM Sean Wallitsch < > sean.wallit...@dreamworks.com> wrote: > > > My apologies for that oversight. My understanding is that many of the > > methods present in aifc depend heavily on audioop for reading

[Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done (was: PEP 594: Removing dead batteries from the standard library)

2019-05-23 Thread Brett Cannon
On Thu, May 23, 2019 at 11:03 AM Barry Warsaw wrote: > On May 20, 2019, at 13:15, Christian Heimes wrote: > > > > here is the first version of my PEP 594 to deprecate and eventually > remove modules from the standard library. The PEP started last year with > talk during the Python Language

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Brett Cannon
On Wed, May 22, 2019 at 1:23 PM Sean Wallitsch < sean.wallit...@dreamworks.com> wrote: > My apologies for that oversight. My understanding is that many of the > methods present in aifc depend heavily on audioop for reading and writing. > But are people using audioop directly? This shifts whether

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Tim Peters
It seems pretty clear now that the primary cause is keeping arenas sorted by number of free pools. As deallocation goes on, the number of distinct "# of free pools" values decreases, leaving large numbers of arenas sharing the same value. Then, e.g., if there are 10,000 arenas with 30 free pools

[Python-Dev] PEP 595: Improving bugs.python.org

2019-05-23 Thread Ezio Melotti
Hello, Berker and I have been working on a PEP that suggests we keep using and improving bugs.python.org and Roundup instead of switching to GitHub Issues as proposed by PEP 581. The PEP covers: * What are the advantages of Roundup over GitHub issues; * What features are missing in Roundup and

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-23 Thread Steve Holden
What would _really_ help is getting the groups that maintain each dead parrot to collaborate on a "Python Legacy release" that adds back anything with a maintainer to the stdlib of the current Python version. Even that will demand large resources, and if it's to be organised in a way that doesn't

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-23 Thread Steve Holden
It might also serve to identify those with an interest in maintaining the non-core packages, which might even be given some special status on PyPI. On Thu, May 23, 2019 at 9:01 AM Alex Walters wrote: > I've watched the entire thread and its taken me a few days to put a finger > on what bothers

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Steve Holden
Besides which, it would be lovely to have a major release that didn't involve any pain at all for the majority of users! Our erstwhile BDFL always eschewed two-digit version identifiers- due to the possibility for confusion about collating sequence, I beleive.. We should honour his preferences

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Gregory P. Smith
I suggest filing a bug to track this... On Thu, May 23, 2019 at 10:13 AM Tim Peters wrote: > [Inada Naoki ] > > ... > > 2. This loop is cleary hot: > > > https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819 > > Which is 3 lines of code

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-23 Thread Barry Warsaw
On May 20, 2019, at 13:15, Christian Heimes wrote: > > here is the first version of my PEP 594 to deprecate and eventually remove > modules from the standard library. The PEP started last year with talk during > the Python Language Summit 2018, https://lwn.net/Articles/755229/. > > The PEP

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Tim Peters
[Inada Naoki ] > ... > 2. This loop is cleary hot: > https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819 Which is 3 lines of code plus a closing brace. The OP didn't build their own Python, and the source from which it was compiled

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Steve Dower
On 23May2019 0947, Anders Munch wrote: Fra: Paul Moore [mailto:p.f.mo...@gmail.com]: A major version change serves as a heads up that something is going on and you need to check the consequences before upgrading. Python's backward compatibility policy allows breaking changes between versions

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Antoine Pitrou
On Fri, 24 May 2019 00:59:08 +0900 Inada Naoki wrote: > > > > It's relatively easy to test replacing our custom allocators with the > > system ones, yes? Can we try those to see whether they have the same > > characteristic? > > > > Yes. > > PYTHONMALLOC=malloc

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Anders Munch
Fra: Paul Moore [mailto:p.f.mo...@gmail.com]: > > A major version change serves as a heads up that something is going on and > > you need to check the consequences before upgrading. > Python's backward compatibility policy allows breaking changes between > versions X.Y and X.Y+1 (with a suitable

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-23 Thread Jeff Kintscher
+1 On 5/23/19 1:00 AM, Alex Walters wrote: I've watched the entire thread and its taken me a few days to put a finger on what bothers me about it. In my opinion, this shouldn't be a pep describing the list of modules that need to go as "dead batteries", but should be a process pep describing

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Inada Naoki
> > It's relatively easy to test replacing our custom allocators with the > system ones, yes? Can we try those to see whether they have the same > characteristic? > Yes. PYTHONMALLOC=malloc LD_PRELOAD=/path/to/jemalloc.so python script.py I will try it tomorrow. >

Re: [Python-Dev] PEP 558: Defined semantics for locals()

2019-05-23 Thread Steve Dower
On 23May2019 0636, Nick Coghlan wrote: However, I think the PR does show that the proposed technique can be implemented without too much additional code complexity, and will hopefully be adaptable for all implementations that emulate the frame API at all. Much excitement! One of the things I

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Steve Dower
On 23May2019 0542, Inada Naoki wrote: 1. perf shows 95% of CPU time is eaten by _PyObject_Free, not kernel space. 2. This loop is cleary hot: https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819 I can attach the process by gdb and I

Re: [Python-Dev] PEP 558: Defined semantics for locals()

2019-05-23 Thread Nick Coghlan
On Wed, 22 May 2019 at 00:51, Nick Coghlan wrote: > P.S. I'm away this weekend, so I expect the reference implementation > to be done late next week, and I'd be submitting the PEP to Nathaniel > for formal pronouncement at that point. However, I'm posting this > thread now so that there's more

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Paul Moore
On Thu, 23 May 2019 at 14:25, Anders Munch wrote: > A major version change serves as a heads up that something is going on and > you need to check the consequences before upgrading. Python's backward compatibility policy allows breaking changes between versions X.Y and X.Y+1 (with a suitable

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Anders Munch
Fra: Python-Dev [mailto:python-dev-bounces+ajm=flonidan...@python.org] På vegne af Terry Reedy: >> Deprecation schedule >> > > I think it worth adding that some modules were deprecated years ago but kept > until after 2.7 end-of-life to ease 3.x transition. Removing modules

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Inada Naoki
1. perf shows 95% of CPU time is eaten by _PyObject_Free, not kernel space. 2. This loop is cleary hot: https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819 I can attach the process by gdb and I confirmed many arenas have same nfreepools.

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Joni Orponen
On Thu, May 23, 2019 at 11:52 AM Inada Naoki wrote: > I have only 32GB mem, but AWS has larger memory machine! > Be aware that on larger cloud vendor instances, the backing vCPUs and memory you get allocated might or might not be spread opaquely across multiple sockets and/or NUMA nodes of the

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Inada Naoki
I have only 32GB mem, but AWS has larger memory machine! Linux perf shows here is bottleneck: https://github.com/python/cpython/blob/master/Objects/obmalloc.c#L1784-L1819 obmalloc sorts arenas by number of free pools. If there are many arenas, and memory block is freed by random order, this

Re: [Python-Dev] Python in next Windows 10 update

2019-05-23 Thread Antoine Pitrou
On Thu, 23 May 2019 00:23:39 +0200 Ray Donnelly wrote: > On Thu, May 23, 2019, 12:17 AM Ivan Pozdeev via Python-Dev < > python-dev@python.org> wrote: > > > On 22.05.2019 23:52, Steve Dower wrote: > > > On 22May2019 1309, Ivan Pozdeev via Python-Dev wrote: > > >> As someone whose job is to

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-23 Thread Alex Walters
I've watched the entire thread and its taken me a few days to put a finger on what bothers me about it. In my opinion, this shouldn't be a pep describing the list of modules that need to go as "dead batteries", but should be a process pep describing how dead batteries should be removed, and the