On 4/8/21 8:46 PM, Les Mikesell wrote:
On Thu, Apr 8, 2021 at 8:25 AM Dave Sherohman <dave.sheroh...@ub.lu.se> wrote:
rsync error: error allocating core memory buffers (code 22) at util2.c(118)
[sender=3.2.0dev]
This is more about the number of files than the size of the drive. Do
you happen to know if there are directories containing millions of
tiny files that could feasibly be archived as a tar or zip file
instead of stored separately?
I don't know details of the filesystem contents on this machine, but our
earlier not-quite-full (some files missed because we didn't use sudo)
backup contained 24,058,239 files according to the bpc host status page,
so that is a possibility.
The synology's admin replied faster than I expected and says there's a
directory where scanned files are dropped which "contains a lot of
files", so I'm looking into whether that can be archived (or skipped -
most of the scans we deal with tend to be temporary files that are
discarded after we do OCR on them).
Also, rsync versions newer than 3.x are supposed to handle it better.
Is your server side extremely old?
The rsync on the synology NAS is version 3.0.9. The bpc server is a
brand-new Debian 11 install, with rsync 3.2.3.
The samba FAQ link mentions that the memory optimization "only works
provided that both sides are 3.0.0 or newer and certain options that
rsync currently can't handle in this mode are not being used." Any idea
what those "certain options" might be? The client-side rsync commands
look pretty basic, so it's probably not using any of them, but it's a
possibility.
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
