On 01/21/2011 11:16 AM, Reuben Thomas wrote: >> Won't this have bad performance on multicore machines, >> > compared to doing the decompression in a separate process? > If it does, you can do the decompression in a separate process.
Sorry, I don't follow. In the common case where I'm doing a 'grep -r', and some files are compressed but others are not, how can I, as a 'grep' user, do the decompression in a separate process? I don't see any easy way to do that. >> > Has the performance of this change been tested recently >> > on modern machines? > No. I suggest trying it out, and doing decompression either in grep (using zlib) or in a separate process (using gzip), before committing to the former implementation. No doubt it depends on the data, but it could be that in typical cases the separate-process approach is faster on modern processors. Also, the separate-process approach is likely to be easier to implement and easier for the user to plug in their own decompressor.
