Phako wrote:
I created 2% fec file of a 14.5 GB tar.lz on a NAS server (4 cores Xeon
E3-1220 V2 with 8 GB of RAM and 8 GB of swap). I didn't time it, but it
took more than 2 hours.
What is the output of 'lziprecover -Fl <fec_file>'?
At default fragmentation level (-9) I guess <fec_file> contains about 1100
FEC blocks of 256 KiB each. This is a lot of work:
http://www.nongnu.org/lzip/manual/lziprecover_manual.html#Implementation-details
"The number of FEC blocks is currently limited to 2048 because of memory and
time limits. Inverting a matrix for 32768 FEC blocks would take a week and
require 2 GiB of RAM. "
You may try fragmentation level -7 or -6.
After that I try to test the resulting files ( lziprecover -Ft ) on a VM
with 8GB of RAM but no swap and the test fails immediately with a memory
allocation error.
What memory allocation error? "Can't mmap: Cannot allocate memory" or "Not
enough memory"?
Lziprecover uses mmap because it needs access to the whole file and "it is
possible to mmap files orders of magnitude larger than both the physical
memory and swap space."[1] Maybe the problem is that the VM you are using
also limits the address space available to mmap.
[1]
https://sourceware.org/glibc/manual/latest/html_node/Memory_002dmapped-I_002fO.html
Does memory requirement also change with the fragmentation level?
Not much. It varies with block size, which is usually small compared with
the size of the protected file.
Best regards,
Antonio.
