On Thu, Dec 24, 2020 at 12:30:35PM +0100, Sebastian Andrzej Siewior wrote:
> On 2020-12-24 00:51:58 [+0300], Vitaly Chikunov wrote:
> > Add guard call to allocate the memory before calling get_thread to
> > prevent memory failures (LZMA_MEM_ERROR).
> > 
> > This would make simple `xz -T0' more robust on 32-bit architectures.
> > 
> > Rationale: simple `-T0' is hard to use portably in scripts on different
> > platforms, because there is always different amount of RAM and CPU,
> > causing unexpected "xz: Cannot allocate memory" errors.
> Could you please say how much CPUs, memory you have and what command
> line you have used?

For example, AND EPYC system is with 32 CPUs (by nproc output) and >200G
memory*. It have no problem using `-T0` in 64-bit mode, but the same system
in PER_LINUX32 personality have 4 GiB memory limit (due to 32-bit address
space), thus, having 32 CPU and just 4GiB memory produces memory
allocation failure. 

You can reproduce the problem even on 64-bit Linux system if you (having
32 cpu) impose 4GiB memory limit by some means, for example like this:
  altair$ nproc
  altair$ ulimit -Sv 4000000
  altair$ head -555444333c /dev/zero | xz -T0 - > /dev/null
  xz: (stdin): Cannot allocate memory

After this patch is applied (test in linux32 on the same system):

  i586:xz-5.2.5$ head -555444333c /dev/zero | src/xz/xz -T0 - > /dev/null


* We use such system for our RPM build/re-build and testing process
in our Linux distribution. But the same scripts (called 'gear') are used
by each maintainer to build packages locally. After we added simple
`-T0` to our xz run (to speed up .txz creation) we are forced to revert
that usage next day, because many packages had rebuild failures on
32-bit systems (we have ARM and i586).

> > Signed-off-by: Vitaly Chikunov <>
> Sebastian

Reply via email to