https://bugzilla.samba.org/show_bug.cgi?id=12769
--- Comment #6 from Dave Gordon <dg32...@zoho.eu> --- The hash table doubles each time it reaches 75% full. A hash table for 32m items @ 16 bytes each (8 byte key, 8 byte void *data) needs 512MB of memory. At the next doubling (up to 64m items) it hits the array allocator limit in utils2.c: #define MALLOC_MAX 0x40000000 void *_new_array(unsigned long num, unsigned int size, int use_calloc) { if (num >= MALLOC_MAX/size) return NULL; return use_calloc ? calloc(num, size) : malloc(num * size); } void *_realloc_array(void *ptr, unsigned int size, size_t num) { if (num >= MALLOC_MAX/size) return NULL; if (!ptr) return malloc(size * num); return realloc(ptr, size * num); } No single array allocation or reallocation is allowed to exceed MALLOC_MAX (1GB). Hence rsync can only handle up to 32m items per invocation if a hash table is required (e.g. for tracking hardlinks when -H is specified). HTH, Dave -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html