[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

[Victor Hooi]

Victor Hooi added the comment:

Hi,

Aha, whoops, sorry Serhiy, didn't see your second message - I think you and I 
posted our last messages at nearly the same time...

Cool, looking forward to your patch =).

Also, is there any chance you could provide a more detailed explanation of 
what's going on? This is just me being curious about how it all works under the 
hood...

Cheers,
Victor

--

___
Python tracker 
<http://bugs.python.org/issue16034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

[Victor Hooi]

Victor Hooi added the comment:

Hi,

I didn't have any buffering size set before, so I believe it defaults to 0 (no 
buffering), right? Wouldn't this be the behaviour on both 2.x and 3.x?

I'm using a 1.5 Mb bzip2 file - I just tried setting buffering to 1000 and 
100, and it didn't seem to make any noticeable difference to the speed of 
reading in the file. E.g.:

f = bz2.BZ2File(filename, 'rb', buffering=100)

What sort of values did you use in relation to your compressed file size to get 
the improvements?

Cheers,
Victor

--

___
Python tracker 
<http://bugs.python.org/issue16034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

[Victor Hooi]

New submission from Victor Hooi:

Hi,

I was writing a script to parse BZ2 blogfiles under Python 2.6, and I noticed 
that bz2file (http://pypi.python.org/pypi/bz2file) seemed to perform much 
slower than with bz2 (native):

http://stackoverflow.com/questions/12575930/is-python-bz2file-slower-than-bz2

I wrote a dummy script that basically just reads through the file, one for bz2 
and one for bz2file (attached):

[vichoo@dev_desktop_vm Desktop]$ time /opt/python3.3/bin/python3.3 testbz2.py > 
/dev/null

real0m5.170s
user0m5.009s
sys 0m0.030s
[vichoo@dev_desktop_vm Desktop]$ time /opt/python3.3/bin/python3.3 
testbz2file.py > /dev/null

real0m5.245s
user0m4.979s
sys 0m0.060s
[vichoo@dev_desktop_vm Desktop]$ time /opt/python2.7/bin/python2.7 testbz2.py > 
/dev/null

real0m0.500s
user0m0.410s
sys 0m0.030s
[vichoo@dev_desktop_vm Desktop]$ time /opt/python2.7/bin/python2.7 
testbz2file.py > /dev/null

real0m5.801s
user0m5.529s
sys 0m0.050s

I also executed "echo 3 > /proc/sys/vm/drop_cache" between each run.

>From this, it appears that Python 2.x's bz2 is fast, but bz2file is slow - and 
>that Python 3.x's bz2 is slow.

Obviously, there could be an issue with the methdology above - however, if not, 
do you know if there are any performance regressions in bz2 from Python 2.x to 
3.x?

Thanks,
Victor

--
components: None
files: testbz2file.py
messages: 171216
nosy: victorhooi
priority: normal
severity: normal
status: open
title: bz2 module appears slower in Python 3.x versus Python 2.x
type: performance
versions: Python 3.3
Added file: http://bugs.python.org/file27288/testbz2file.py

___
Python tracker 
<http://bugs.python.org/issue16034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com