On 2016-01-30, Lampshade <[email protected]> wrote:
> xz: Adjusted the number of threads from 2 to 1 to not exceed the memory
> usage limit of 1600 MiB
>
> 1600 is clearly larger than 674*2=1348
A closer reading of the man page reveals that memory consumption
is even higher in multi-threaded mode.
In multi-threaded mode about three times _size_ bytes will be
allocated in each thread for buffering input and output. The
default _size_ is three times the LZMA2 dictionary size or 1 MiB,
whichever is more.
At -9, the dictionary size is 64 MB, so this adds another 576 MB
per thread. A quick check with top(1) confirms that xz in
multi-threaded mode allocates 1250 MB per thread at compression
level 9.
> In the end I can compress, but I think that something is wrong.
The last relevant change is that we actually enabled multi-threading.
Before that, the -T option was silently ignored. However that
change was ten months ago and is already in the 5.8 packages.
Anyway, there is nothing wrong, and as I said before:
You need to raise the limit or use a less greedy compression setting.
--
Christian "naddy" Weisgerber [email protected]
