On 2013-06-26 Alexander Clouter wrote: > Attached is a patch that enables 'streaming' support for xz output, > in short LZMA_SYNC_FLUSH is called every X milliseconds.
There is now this kind of feature in the git repostory that can be tested. I named the option --flush-timeout=TIMEOUT where the timeout is in milliseconds. In contrast to your patch, the committed code calls read() as long as read() can fill the buffer completely. poll() is only called when read() would block and only then is the flush-timeout checked. Thus, the system time isn't polled with clock_gettime() or gettimeofday() on every call to io_read() when the flush-timeout is active. The --long-help or the man page haven't been updated yet. It is possible that this feature isn't in its final form yet. > We find it > helpful so that we can effectively do: > ---- > tail -f foobar.log.xz | nc w.x.y.z 1234 > ---- > > Meanwhile foobar.log.xz is effectively being generated with: > ---- > tail -f foobar.log | xz -c --select-timeout 500 > foobar.log.xz > ---- > > This means the receiver then gets something that is decodeable in X > milliseconds rather than having to wait for a whole block to be > generated and flushed, which might be a considerable time if whatever > is writing to foobar.log is low volume (100 bytes per second for > example). For now, xz cannot be used for the decompression side because xz does too much buffering. It is similar with XZ for Java unless one reads one byte at a time. xz should naturally be usable for the decompression side too. I haven't decided yet how to fix it (e.g. require an option or perhaps always disable buffering). -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
