Hi!
I'm working on an application that uses xz for data compression. I'll simplify
it a bit, but basically I need to send files over the net, and I compress them
using xz because I think the less data I need to send over the net, the better.
Now the problem is that for those files I cannot predict the size. Often they
will be quite small, but they also could be 100 MB in size or more. So I use
xz -9 to get the best compression.
The problem is now that xz takes a lot of time to start:
stefan@ubuntu:/tmp$ time echo "foo" | xz -9 >/dev/null
real 0m0.155s
user 0m0.052s
sys 0m0.096s
Probably this is some initialization code: oprofile says that xz doesn't do
much itself, but spends the time in libc:
60544 26.1879 xz
CPU_CLK_UNHALT...|
samples| %|
------------------
59902 98.9396 libc-2.13.so
439 0.7251 ld-2.13.so
139 0.2296 liblzma.so.2.0.0
38 0.0628 xz
26 0.0429 libpthread-2.13.so
and it seems that all time is spent with memset:
59376 25.6834 libc-2.13.so xz
__memset_x86_64
Would it be possible to not allocate/initialize all the memory at startup? Then
the time for one single xz invocation could certainly be improved quite a bit,
at
least if the data to be compressed is small.
Cu... Stefan
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
