Re: [Python-Dev] Prefetching on buffered IO files

2010-09-29 Thread Hagen Fürstenau
Ow... I've always assumed that seek() is essentially free, because that's how a typical OS kernel implements it. If seek() is bad on GzipFile, how hard would it be to fix this? I'd imagine that there's no easy way to make arbitrary seeks on a GzipFile fast. But wouldn't it be enough to

Re: [Python-Dev] Prefetching on buffered IO files

2010-09-29 Thread Antoine Pitrou
On Wed, 29 Sep 2010 10:06:57 +0200 Hagen Fürstenau ha...@zhuliguan.net wrote: Ow... I've always assumed that seek() is essentially free, because that's how a typical OS kernel implements it. If seek() is bad on GzipFile, how hard would it be to fix this? I'd imagine that there's no easy

[Python-Dev] Prefetching on buffered IO files

2010-09-27 Thread Antoine Pitrou
Hello, While trying to solve #3873 (poor performance of pickle on file objects, due to the overhead of calling read() with very small values), it occurred to me that the prefetching facilities offered by BufferedIOBase are not flexible and efficient enough. Indeed, if you use seek() and read(),

Re: [Python-Dev] Prefetching on buffered IO files

2010-09-27 Thread Guido van Rossum
On Mon, Sep 27, 2010 at 3:41 PM, Antoine Pitrou solip...@pitrou.net wrote: While trying to solve #3873 (poor performance of pickle on file objects, due to the overhead of calling read() with very small values), it occurred to me that the prefetching facilities offered by BufferedIOBase are not