Good day.
I have installed Python 3 and i have a problem with the builtin read()
function.
[code]
huge = open ( 'C:/HUGE_FILE.pcl', 'rb', 0 )
import io
vContent = io.StringIO()
vContent = huge.read() # This line takes hours to process !!!
vSplitContent = vContent.split
(
Cro wrote:
Good day.
I have installed Python 3 and i have a problem with the builtin read()
function.
[code]
huge = open ( 'C:/HUGE_FILE.pcl', 'rb', 0 )
import io
vContent = io.StringIO()
vContent = huge.read() # This line takes hours to process !!!
vSplitContent = vContent.split
(
Do you really mean io.StringIO? I guess you want io.BytesIO() ..
Christian
Mmm... i don't know.
I also tried :
[code]
IDLE 3.0
import io
vContent = io.BytesIO()
huge = io.open(C:\HUGE_FILE.pcl,'r+b',0)
vContent = huge.read()
[/code]
It still waits a lot... i don't have the patience to
On Thu, Dec 4, 2008 at 11:48 AM, Christian Heimes [EMAIL PROTECTED] wrote:
Cro wrote:
vContent = io.StringIO()
vContent = huge.read() # This line takes hours to process !!!
Do you really mean io.StringIO? I guess you want io.BytesIO() ..
I don't think it matters. Here's a quick comparison
On Thu, Dec 4, 2008 at 8:57 AM, Cro [EMAIL PROTECTED] wrote:
Do you really mean io.StringIO? I guess you want io.BytesIO() ..
Christian
Mmm... i don't know.
I also tried :
[code]
IDLE 3.0
import io
vContent = io.BytesIO()
You do realize that the previous line is completely pointless,
huge = io.open(C:\HUGE_FILE.pcl,'r+b',0)
Why do you want to disable buffering? From the io.open help:
open(file, mode='r', buffering=None, encoding=None, errors=None,
newline=None, closefd=True)
Open file and return a stream. Raise IOError upon failure.
...
buffering is
Cro wrote:
Good day.
I have installed Python 3 and i have a problem with the builtin read()
function.
[code]
huge = open ( 'C:/HUGE_FILE.pcl', 'rb', 0 )
import io
vContent = io.StringIO()
vContent = huge.read() # This line takes hours to process !!!
vSplitContent = vContent.split
(
I can confirm this,
I am getting very slow read performance when reading a smaller 20 MB
file.
- Python 2.5 takes 0.4 seconds
- Python 3.0 takes 62 seconds
fname = dmel-2R-chromosome-r5.1.fasta
data = open(fname, 'rt').read()
print ( len(data) )
--
Jerry Hill wrote:
On Thu, Dec 4, 2008 at 11:48 AM, Christian Heimes [EMAIL PROTECTED] wrote:
Cro wrote:
vContent = io.StringIO()
vContent = huge.read() # This line takes hours to process !!!
Do you really mean io.StringIO? I guess you want io.BytesIO() ..
I don't think it matters. Here's a
On Dec 4, 1:31 pm, Terry Reedy [EMAIL PROTECTED] wrote:
Jerry Hill wrote:
That's 3 orders of magnitude slower on python3.0!
Timing of os interaction may depend on os. I verified above on WinXp
with 4 meg Pythonxy.chm file. Eye blink versus 3 secs, duplicated. I
think something is wrong
I don't think it matters. Here's a quick comparison between 2.5 and
3.0 on a relatively small 17 meg file:
C:\c:\Python30\python -m timeit -n 1
open('C:\\work\\temp\\bppd_vsub.csv', 'rb').read()
1 loops, best of 3: 36.8 sec per loop
C:\c:\Python25\python -m timeit -n 1
On Dec 4, 2:01 pm, Дамјан Георгиевски [EMAIL PROTECTED] wrote:
I don't think it matters. Here's a quick comparison between 2.5 and
3.0 on a relatively small 17 meg file:
C:\c:\Python30\python -m timeit -n 1
open('C:\\work\\temp\\bppd_vsub.csv', 'rb').read()
1 loops, best of 3: 36.8 sec
Дамјан Георгиевски wrote:
I don't think it matters. Here's a quick comparison between 2.5 and
3.0 on a relatively small 17 meg file:
C:\c:\Python30\python -m timeit -n 1
open('C:\\work\\temp\\bppd_vsub.csv', 'rb').read()
1 loops, best of 3: 36.8 sec per loop
C:\c:\Python25\python -m timeit -n
Terry Reedy wrote:
Дамјан Георгиевски wrote:
I don't think it matters. Here's a quick comparison between 2.5 and
3.0 on a relatively small 17 meg file:
C:\c:\Python30\python -m timeit -n 1
open('C:\\work\\temp\\bppd_vsub.csv', 'rb').read()
1 loops, best of 3: 36.8 sec per loop
On Thu, 04 Dec 2008 14:25:48 -0500, Terry Reedy [EMAIL PROTECTED] wrote:
[snip]
In my test, I read Python25.chm with 2.5 and Python30.chm with 3.0.
Rereading Python30.chm without closing *is* much faster.
f=open('Doc/Python30.chm','rb')
d=f.read()
d=f.read()
d=f.read()
Did you think
On Thu, 2008-12-04 at 20:01 +0100, Дамјан Георгиевски wrote:
I don't think it matters. Here's a quick comparison between 2.5 and
3.0 on a relatively small 17 meg file:
C:\c:\Python30\python -m timeit -n 1
open('C:\\work\\temp\\bppd_vsub.csv', 'rb').read()
1 loops, best of 3: 36.8 sec
Jean-Paul Calderone wrote:
On Thu, 04 Dec 2008 14:25:48 -0500, Terry Reedy [EMAIL PROTECTED] wrote:
[snip]
In my test, I read Python25.chm with 2.5 and Python30.chm with 3.0.
Rereading Python30.chm without closing *is* much faster.
f=open('Doc/Python30.chm','rb')
d=f.read()
d=f.read()
Turns out write performance is also slow!
The program below takes
3 seconds on python 2.5
17 seconds on python 3.0
yes, 17 seconds! tested many times in various order. I believe the
slowdowns are not constant (3x) but some sort of nonlinear function
(quadratic?) play with the N to see it.
Terry Reedy wrote:
Timing of os interaction may depend on os. I verified above on WinXp
with 4 meg Pythonxy.chm file. Eye blink versus 3 secs, duplicated. I
think something is wrong that needs fixing in 3.0.1.
http://bugs.python.org/issue4533
I've attached a patch to the bug. reading was
Christian Heimes wrote:
Terry Reedy wrote:
Timing of os interaction may depend on os. I verified above on WinXp
with 4 meg Pythonxy.chm file. Eye blink versus 3 secs, duplicated. I
think something is wrong that needs fixing in 3.0.1.
http://bugs.python.org/issue4533
I've attached a
20 matches
Mail list logo
