Re: [Python-Dev] Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?

2017-07-18 Thread Nathaniel Smith
I'd probably start with a regular C-level profiler, like perf or callgrind. They're not very useful for comparing two versions of code written in Python, but here the Python code is the same (modulo changes in the stdlib), and it's changes in the interpreter's C code that probably make the differen

Re: [Python-Dev] Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?

2017-07-18 Thread Ben Hoyt
Thanks, Nick -- that's interesting. I just saw the extra JUMP_FORWARD and JUMP_ABSOLUTE instructions on my commute home (I guess those are something Python 3.x optimizes away). VERY strangely, on Windows Python 2.7 is faster! Comparing 64-bit Python 2.7.12 against Python 3.5.3 on my Windows 10 lap

Re: [Python-Dev] Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?

2017-07-18 Thread Nick Coghlan
On 19 July 2017 at 02:18, Antoine Pitrou wrote: > On Tue, 18 Jul 2017 12:03:36 -0400 > Ben Hoyt wrote: >> The program is a pentomino puzzle solver, and it works via code generation, >> generating a ton of nested "if" statements, so I believe it's exercising >> the Python bytecode interpreter heav

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Brett Cannon
On Tue, 18 Jul 2017 at 09:07 Ethan Furman wrote: > Raymond Hettinger: > - > > One minor grumble: I think we need to give careful cost/benefit > considerations to > > optimizations that complicate the implementation. Over the last > several years, the > > source for Python has

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread Larry Hastings
On 07/17/2017 07:25 PM, Nathaniel Smith wrote: I think the intention is to allow users with a certain kind of security requirement to opt in to a restricted version of the language that doesn't support exec. This is difficult if the stdlib is calling exec all over the place. But nobody is sug

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread Brett Cannon
On Mon, 17 Jul 2017 at 19:26 Nathaniel Smith wrote: > On Jul 17, 2017 5:28 PM, "Steven D'Aprano" wrote: > > On Mon, Jul 17, 2017 at 09:31:20PM +, Brett Cannon wrote: > > > As for removing exec() as a goal, I'll back up Christian's point and the > > one Steve made at the language summit that

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Ethan Furman
On 07/18/2017 09:16 AM, Antoine Pitrou wrote: On Tue, 18 Jul 2017 09:08:08 -0700 Ethan Furman wrote: Nick Coughlan: - It is "Nick Coghlan" not "Coughlan". Argh. Sorry, Nick, and thank you, Antoine! As another example of this: while trading the global import lock for per-modu

Re: [Python-Dev] Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?

2017-07-18 Thread Antoine Pitrou
On Tue, 18 Jul 2017 12:03:36 -0400 Ben Hoyt wrote: > Hi folks, > > (Not entirely sure this is the right place for this question, but hopefully > it's of interest to several folks.) > > A few days ago I posted a note in response to Victor Stinner's articles on > his CPython contributions, noting

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Victor Stinner
2017-07-18 18:08 GMT+02:00 Ethan Furman : > Raymond Hettinger: > - >> And complexity leads to bugs >> (the C >> optimization of random number seeding caused a major bug in the 3.6.0 >> release Hum, I guess that Raymond is referring to http://bugs.python.org/issue29085 This regress

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Antoine Pitrou
On Tue, 18 Jul 2017 09:08:08 -0700 Ethan Furman wrote: > > Nick Coughlan: > - It is "Nick Coghlan" not "Coughlan". > > As another example of this: while trading the global import lock for > > per-module locks eliminated most of the old import deadlocks, it turns > > out that it *als

Re: [Python-Dev] deque implementation question

2017-07-18 Thread Joao S. O. Bueno
On 15 July 2017 at 04:01, Max Moroz wrote: > What would be the disadvantage of implementing collections.deque as a > circular array (rather than a doubly linked list of blocks)? My naive > thinking was that a circular array would maintain the current O(1) append/pop > from either side, and would

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Victor Stinner
[Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability 2017-07-18 18:08 GMT+02:00 Ethan Furman : > Nick Coughlan: > - >> >> As another example of this: while trading the global import lock for >> per-module locks eliminated most of the old import deadlocks, (...) Mi

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread Ethan Furman
On 07/18/2017 08:12 AM, Guido van Rossum wrote: There are some weighty things being said in this subthread that shouldn't be hidden under the heading of improving NamedTuple. For continued discussion of our development philosophy let's open a new thread. (I have an opinion but I expect I'm not

[Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Ethan Furman
Raymond Hettinger: - > One minor grumble: I think we need to give careful cost/benefit considerations to > optimizations that complicate the implementation. Over the last several years, the > source for Python has grown increasingly complicated. Fewer people understand it > n

[Python-Dev] Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?

2017-07-18 Thread Ben Hoyt
Hi folks, (Not entirely sure this is the right place for this question, but hopefully it's of interest to several folks.) A few days ago I posted a note in response to Victor Stinner's articles on his CPython contributions, noting that I wrote a program that ran in 11.7 seconds on Python 2.7, but

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread Guido van Rossum
There are some weighty things being said in this subthread that shouldn't be hidden under the heading of improving NamedTuple. For continued discussion of our development philosophy let's open a new thread. (I have an opinion but I expect I'm not the only one with that opinion, so I'll let others e

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread Antoine Pitrou
On Tue, 18 Jul 2017 22:13:01 +1000 Nick Coghlan wrote: > > As another example of this: while trading the global import lock for > per-module locks eliminated most of the old import deadlocks, it turns > out that it *also* left us with some fairly messy race conditions and > more fragile code (I s

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread Nick Coghlan
On 18 July 2017 at 05:42, Raymond Hettinger wrote: > One minor grumble: I think we need to give careful cost/benefit > considerations to optimizations that complicate the implementation. Over the > last several years, the source for Python has grown increasingly complicated. > Fewer people u

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread INADA Naoki
> > I think that the right solution of this issue is generalizing the import > machinery and allowing it to cache not just files, but arbitrary chunks of > code. We already use precompiled bytecode files for exactly same goal -- > speed up the startup by avoiding compilation. This solution could be