Hi: I'm in the middle of recoverying from a tactical error copying around an Mac OS X 10.10.5 Time Machine backup (turns out Apple's instructions aren't great...), and I had rsync running for the past 6 hours repairing permissions/acls on 1.5 TB of data (not copying the data), and then it just died in the middle with:
.L....og.... 2015-03-11-094807/platinum-bar2/usr/local/mysql -> mysql-5.6.21-osx 10.8-x86_64 ERROR: out of memory in expand_item_list [sender] rsync error: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2] rsync: [sender] write error: Broken pipe (32) It was invoked as rsync -iaHJAX platinum-barzoom/Backups.backupdb/pb3/ platinum-barratry/x/Backups.backupdb/pb3/ I suspect the situation will be different the next time around, and I'm also not really inclined to try to wait another 7 hours for it to fail. Process limits were: bash-3.2# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 256 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 709 virtual memory (kbytes, -v) unlimited Based on "df -i", there are about 200 million inodes in the scope of the copy bash-3.2# df -i Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted on /dev/disk2s2 3906353072 3879969656 26383416 100% 484996205 3297927 99% /Volumes/platinum-barzoom /dev/disk1s2 9766869344 3327435312 6439434032 35% 207964705 402464627 34% /Volumes/platinum-barratry Because this is a Time Machine backup, and there were 66 snapshots of a 1 TB disk consuming about 1.5 TB, there were a *lot* of hard links. Many of directories rather than individual files, so it's a little challenging to estimate what the number of files to links is. Are there any useful tips here? Is it worth filing a bug report on this thin record? I guess I can turn on core dumps and increase (unlimit completely) the stack size... Although it doesn't seem to have segfaulted, so I'm not sure having core dumps enabled would have helped? This was rsync 3.1.2 as installed via Homebrew which appears to have applied 3 local patches: https://github.com/Homebrew/homebrew-core/blob/68fe052398ea0939315ad87b0a08313c9d9a95af/Formula/rsync.rb apply "patches/fileflags.diff", "patches/crtimes.diff", "patches/hfs-compression.diff" p.s.: If I had to start over, I would have spent less time just deleting the data and recopying it, rather than trying to fixup the metadata and dealing with magic Apple stuff like the inability to modify symlinks inside a top-level Backups.backupdb directory of a Time Machine hfs volume (But you can move the top-level directory into another directory and then modify symlinks inside and then move it back). This has been an "interesting" experience. Thanks. --jh...@mit.edu John Hawkinson -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html