Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Serhiy Storchaka
On 29.01.16 19:05, Steve Dower wrote: This is probably the code snippet that bothered me the most: ### Encoding table encoding_table=codecs.charmap_build(decoding_table) It shows up in many of the encodings modules, and while it is not a bad function in itself, we are obviously genera

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Oscar Benjamin
On 30 January 2016 at 03:48, Steve Dower wrote: > > It doesn't currently end up on disk. Some tables are partially or completely > stored on disk as Python source code (some are partially generated from > simple rules), but others are generated by inverting those. That process > takes time that co

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Steve Dower
On 30Jan2016 0645, Serhiy Storchaka wrote: $ ./python -m timeit -s "import codecs; from encodings.cp437 import decoding_table" -- "codecs.charmap_build(decoding_table)" 10 loops, best of 3: 4.36 usec per loop Getting rid from charmap_build() would save you at most 4.4 microseconds per encodi

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Serhiy Storchaka
On 30.01.16 18:31, Steve Dower wrote: On 30Jan2016 0645, Serhiy Storchaka wrote: $ ./python -m timeit -s "import codecs; from encodings.cp437 import decoding_table" -- "codecs.charmap_build(decoding_table)" 10 loops, best of 3: 4.36 usec per loop Getting rid from charmap_build() would save

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Brett Cannon
On Sat, 30 Jan 2016 at 10:21 Serhiy Storchaka wrote: > On 30.01.16 18:31, Steve Dower wrote: > > On 30Jan2016 0645, Serhiy Storchaka wrote: > >> $ ./python -m timeit -s "import codecs; from encodings.cp437 import > >> decoding_table" -- "codecs.charmap_build(decoding_table)" > >> 10 loops, be

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Steve Dower
Brett tried freezing the entire stdlib at one point (as we do for parts of importlib) and reported no significant improvement. Since that rules out code compilation as well as the OS calls, it'd seem the priority is to execute less code on startup. Details of that work were posted to python-dev

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Brett Cannon
On Sat, Jan 30, 2016, 12:30 Sven R. Kunze wrote: > On 30.01.2016 19:20, Serhiy Storchaka wrote: > > AFAIK the most time is spent in system calls like stat or open. > > Archiving the stdlib into the ZIP file and using zipimport can > > decrease Python startup time (perhaps there is an open issue a

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Sven R. Kunze
On 30.01.2016 19:20, Serhiy Storchaka wrote: AFAIK the most time is spent in system calls like stat or open. Archiving the stdlib into the ZIP file and using zipimport can decrease Python startup time (perhaps there is an open issue about this). Oh, please don't. One thing I love about Python

Re: [Python-Dev] More optimisation ideas

2016-01-30 Thread Sven R. Kunze
On 30.01.2016 21:32, Brett Cannon wrote: On Sat, Jan 30, 2016, 12:30 Sven R. Kunze > wrote: On 30.01.2016 19:20, Serhiy Storchaka wrote: > AFAIK the most time is spent in system calls like stat or open. > Archiving the stdlib into the ZIP file and using zipimp