Re: [Python-Dev] Prefetching on buffered IO files

2010-09-29 Thread Antoine Pitrou
On Wed, 29 Sep 2010 10:06:57 +0200 Hagen Fürstenau wrote: > > Ow... I've always assumed that seek() is essentially free, because > > that's how a typical OS kernel implements it. If seek() is bad on > > GzipFile, how hard would it be to fix this? > > I'd imagine that there's no easy way to make a

Re: [Python-Dev] Prefetching on buffered IO files

2010-09-29 Thread Hagen Fürstenau
> Ow... I've always assumed that seek() is essentially free, because > that's how a typical OS kernel implements it. If seek() is bad on > GzipFile, how hard would it be to fix this? I'd imagine that there's no easy way to make arbitrary seeks on a GzipFile fast. But wouldn't it be enough to optim

Re: [Python-Dev] Prefetching on buffered IO files

2010-09-27 Thread Guido van Rossum
On Mon, Sep 27, 2010 at 3:41 PM, Antoine Pitrou wrote: > While trying to solve #3873 (poor performance of pickle on file > objects, due to the overhead of calling read() with very small values), > it occurred to me that the prefetching facilities offered by > BufferedIOBase are not flexible and ef

[Python-Dev] Prefetching on buffered IO files

2010-09-27 Thread Antoine Pitrou
Hello, While trying to solve #3873 (poor performance of pickle on file objects, due to the overhead of calling read() with very small values), it occurred to me that the prefetching facilities offered by BufferedIOBase are not flexible and efficient enough. Indeed, if you use seek() and read(),