On 2013-10-27 Richard W.M. Jones wrote: > On Sat, Oct 26, 2013 at 08:06:45PM +0300, Lasse Collin wrote: > > > - For reasons I don't understand, both regular xzcat and pxzcat > > > cause the output file to be flushed to disk after the program > > > exits. This causes any program which consumes the output of the > > > file to slow down. > > > > I have no idea. I see you committed something that seems to be > > related to this after your email. With a quick reading I don't > > understand it well, it seems to be working around some issue with > > ftruncate() with ext4. > > That's right. It turned out to be a misfeature in ext4: if you > truncate a file from a non-zero size down to a zero size, ext4 flags > the file and flushes it on close. ("Truncate" includes both O_TRUNC > and ftruncate). This can be disabled with the noauto_da_alloc mount > option, but of course that is not the default.
If I remember correctly, that "misfeature" was added because there are too many programs that write config files by overwriting the old files with O_TRUNC. By flushing quickly ext4 tries to avoid zero-length config files after a crash or a power failure. With xz there's nothing I can do to work around the flushes (and I'm not sure if I wanted do anything for this even if I could). The truncation is done by the shell when one redirects the output from xz to a file, so to avoid the flushes one would need to patch the shell. -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode