[Python-Dev] Pure pickle bechmark.
Hello,
1).I was going through the code of *python pickle* to search any
optimization possibility.But the only thing that I found very alarming was
again the import time(I tried with lazy-import but it didn't helped much.)
I found py3 to be ~45 times slower on* initial imports(very raw
measure..using "time." ) *as compared to py2 on an usual example.
py3->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.009715557098388672(time taken to do initial imports...measured using
*time.time()* )
py2->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.000236034393311(time taken to do initial imports...measured using
*time.time()* )
Do you have any thought/ideas on improving this?
Thank You.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pure pickle bechmark.
On 2017-07-09 15:08, Bhavishya wrote:
Hello,
1).I was going through the code of *python pickle* to search any
optimization possibility.But the only thing that I found very alarming
was again the import time(I tried with lazy-import but it didn't helped
much.)
I found py3 to be ~45 times slower on*initial imports(very raw
measure..using "time." ) *as compared to py2 on an usual example.
py3->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.009715557098388672(time taken to do initial imports...measured using
*time.time()* )
py2->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.000236034393311(time taken to do initial imports...measured using
*time.time()* )
Do you have any thought/ideas on improving this?
Python 3 is using Unicode strings, whereas Python 2 is using bytestrings.
What you show above are very short (in time) examples (less than 1/100
of a second), so they're not that meaningful.
If you had timed pickling a substantial object (the same object in both
cases) and it took a significant amount of time and you found a
significant slowdown, then it would be worth looking into further.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pure pickle bechmark.
Hi, On Sun, 9 Jul 2017 19:38:09 +0530 Bhavishya wrote: > Hello, > > 1).I was going through the code of *python pickle* to search any > optimization possibility.But the only thing that I found very alarming was > again the import time(I tried with lazy-import but it didn't helped much.) > > I found py3 to be ~45 times slower on* initial imports(very raw > measure..using "time." ) *as compared to py2 on an usual example. Can you explain how you measured exactly? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pure pickle bechmark.
I don't know this is relating to your case.
When I saw Victor's report [1], I researched why Python 3 is slower than
Python 2 on unpickle_pure_python benchmark.
[1] https://mail.python.org/pipermail/speed/2017-February/000503.html
And I found Python 2 and 3 uses different version of pickle format.
Current Python 3 uses "framing" format. While unpickling, `read(1)` is
very performance critical. Python 2 uses `cStringIO.read` which is
implemented in C.
On the other hand, Python 3 uses `_Unframer.read` which is implemented
in Python.
Since this is not relating to "first import time", I don't know this
is what you want to optimize.
(Since _pickle is used for normal case, pure Python unpickle
performance is not a common
problem).
If you want to optimize it, _Unframer uses BytesIO internally and
performance critical
part may be able to call BytesIO.read directly instead of _Unframer.read.
Regards,
INADA Naoki
On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya wrote:
> Hello,
>
> 1).I was going through the code of python pickle to search any optimization
> possibility.But the only thing that I found very alarming was again the
> import time(I tried with lazy-import but it didn't helped much.)
>
> I found py3 to be ~45 times slower on initial imports(very raw
> measure..using "time." ) as compared to py2 on an usual example.
>
> py3->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.009715557098388672(time taken to do initial imports...measured using
> time.time() )
>
> py2->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.000236034393311(time taken to do initial imports...measured using
> time.time() )
>
> Do you have any thought/ideas on improving this?
>
>
> Thank You.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pure pickle bechmark.
Wait. Are we talking about the C accelerator or the pure Python
implementation of pickle on Python 3?
Victor
Le 10 juil. 2017 01:19, "INADA Naoki" a écrit :
> I don't know this is relating to your case.
>
> When I saw Victor's report [1], I researched why Python 3 is slower than
> Python 2 on unpickle_pure_python benchmark.
>
> [1] https://mail.python.org/pipermail/speed/2017-February/000503.html
>
>
> And I found Python 2 and 3 uses different version of pickle format.
>
> Current Python 3 uses "framing" format. While unpickling, `read(1)` is
> very performance critical. Python 2 uses `cStringIO.read` which is
> implemented in C.
> On the other hand, Python 3 uses `_Unframer.read` which is implemented
> in Python.
>
> Since this is not relating to "first import time", I don't know this
> is what you want to optimize.
> (Since _pickle is used for normal case, pure Python unpickle
> performance is not a common
> problem).
>
> If you want to optimize it, _Unframer uses BytesIO internally and
> performance critical
> part may be able to call BytesIO.read directly instead of _Unframer.read.
>
> Regards,
> INADA Naoki
>
>
> On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya
> wrote:
> > Hello,
> >
> > 1).I was going through the code of python pickle to search any
> optimization
> > possibility.But the only thing that I found very alarming was again the
> > import time(I tried with lazy-import but it didn't helped much.)
> >
> > I found py3 to be ~45 times slower on initial imports(very raw
> > measure..using "time." ) as compared to py2 on an usual example.
> >
> > py3->
> > ./python -c '
> > favorite_color = { "lion": "yellow", "kitty": "red" }
> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> > 0.009715557098388672(time taken to do initial imports...measured using
> > time.time() )
> >
> > py2->
> > ./python -c '
> > favorite_color = { "lion": "yellow", "kitty": "red" }
> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> > 0.000236034393311(time taken to do initial imports...measured using
> > time.time() )
> >
> > Do you have any thought/ideas on improving this?
> >
> >
> > Thank You.
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> victor.stinner%40gmail.com
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pure pickle bechmark.
I said about pure Python implementation (unpickle_pure_python),
because mail title is "Pure pickle bechmark".
INADA Naoki
On Mon, Jul 10, 2017 at 8:36 AM, Victor Stinner
wrote:
> Wait. Are we talking about the C accelerator or the pure Python
> implementation of pickle on Python 3?
>
> Victor
>
> Le 10 juil. 2017 01:19, "INADA Naoki" a écrit :
>>
>> I don't know this is relating to your case.
>>
>> When I saw Victor's report [1], I researched why Python 3 is slower than
>> Python 2 on unpickle_pure_python benchmark.
>>
>> [1] https://mail.python.org/pipermail/speed/2017-February/000503.html
>>
>>
>> And I found Python 2 and 3 uses different version of pickle format.
>>
>> Current Python 3 uses "framing" format. While unpickling, `read(1)` is
>> very performance critical. Python 2 uses `cStringIO.read` which is
>> implemented in C.
>> On the other hand, Python 3 uses `_Unframer.read` which is implemented
>> in Python.
>>
>> Since this is not relating to "first import time", I don't know this
>> is what you want to optimize.
>> (Since _pickle is used for normal case, pure Python unpickle
>> performance is not a common
>> problem).
>>
>> If you want to optimize it, _Unframer uses BytesIO internally and
>> performance critical
>> part may be able to call BytesIO.read directly instead of _Unframer.read.
>>
>> Regards,
>> INADA Naoki
>>
>>
>> On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya
>> wrote:
>> > Hello,
>> >
>> > 1).I was going through the code of python pickle to search any
>> > optimization
>> > possibility.But the only thing that I found very alarming was again the
>> > import time(I tried with lazy-import but it didn't helped much.)
>> >
>> > I found py3 to be ~45 times slower on initial imports(very raw
>> > measure..using "time." ) as compared to py2 on an usual example.
>> >
>> > py3->
>> > ./python -c '
>> > favorite_color = { "lion": "yellow", "kitty": "red" }
>> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
>> > 0.009715557098388672(time taken to do initial imports...measured using
>> > time.time() )
>> >
>> > py2->
>> > ./python -c '
>> > favorite_color = { "lion": "yellow", "kitty": "red" }
>> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
>> > 0.000236034393311(time taken to do initial imports...measured using
>> > time.time() )
>> >
>> > Do you have any thought/ideas on improving this?
>> >
>> >
>> > Thank You.
>> ___
>> Python-Dev mailing list
>> [email protected]
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pure pickle bechmark.
Please explain how to reproduce your benchmark. Maybe write a shell script?
Victor
Le 9 juil. 2017 17:49, "Bhavishya" a écrit :
> Hello,
>
> 1).I was going through the code of *python pickle* to search any
> optimization possibility.But the only thing that I found very alarming was
> again the import time(I tried with lazy-import but it didn't helped much.)
>
> I found py3 to be ~45 times slower on* initial imports(very raw
> measure..using "time." ) *as compared to py2 on an usual example.
>
> py3->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.009715557098388672(time taken to do initial imports...measured using
> *time.time()* )
>
> py2->
> ./python -c '
> favorite_color = { "lion": "yellow", "kitty": "red" }
> pickle.dump( favorite_color, open( "save.p", "wb" ) )'
> 0.000236034393311(time taken to do initial imports...measured using
> *time.time()* )
>
> Do you have any thought/ideas on improving this?
>
>
> Thank You.
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> victor.stinner%40gmail.com
>
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pure pickle bechmark.
(Oops, I didn't notice that we started to talk off the list, let's discuss that in python-dev please.) I don't see the point of optimizing "pickle/unpickle pure python" benchmark on Python 3. This benchmark doesn't make sense on Python 3, since I don't know anyone using the pure Python pickle. The C accelerator is now used by default. I already proposed to remove this benchmark: https://mail.python.org/pipermail/speed/2017-April/000554.html *but* Antoine Pitrou mentionned that the cloudpickle project uses it. Maybe we should try to understand what's wrong with _pickle (C module) for cloudpickle? Victor 2017-07-10 2:10 GMT+02:00 Bhavishya : > I was working on the two regressed benchmarks (i.e. pickle/unpickle > pure-python), and as it was a case with other benchmarksthat performance > is affected by import ...I thought that could be a case with pickle.py > too. And thus tried adding the above patch to Lib/pickle.py to measure the > initial import time. > > I haven't tried it for any practical use-case. > > > On Mon, Jul 10, 2017 at 5:27 AM, Victor Stinner > wrote: >> >> Sorry, I don't understand the direct link between the import time of 4 >> modules and the pickle module. Can you please elaborate? >> >> What are you trying to optimize? >> >> What is your use case? >> >> Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
