> I guess I can turn on core dumps and increase (unlimit completely) the
> stack size...
>
> Although it doesn't seem to have segfaulted, so I'm not sure having
> core dumps enabled would have helped?
Indeed. I reran it on the 50 remaining individual directories,
it seems to have made it through 10 before failing again overnight:
66596 20:38:03 rsync -vP -iaHJAX `cat /tmp/unback`
/Volumes/platinum-barratry/x/Backups.backupdb/pb3/
...
rsync: unpack_smb_acl: sys_acl_get_info(): Undefined error: 0 (0)
rsync: unpack_smb_acl: sys_acl_get_info(): Undefined error: 0 (0)
...
.f....og.a..
2015-06-19-072520/platinum-bar2/Users/jhawk/.emacs.d/auto-save-list/.saves-1541-platinum-bar2.local~
ERROR: out of memory in expand_item_list [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106)
[sender=3.1.2]
bash-3.2#
No segfault, no coredump, no syslogs.
Do I need to run this under lldb and set a breakpoint in expand_item_list()?
Quick inspection suggests running with -vvvv might give some useful output:
1680 if (DEBUG_GTE(FLIST, 3)) {
1681 rprintf(FINFO, "[%s] expand %s to %s bytes, did%s move\n",
1682 who_am_i(), desc, big_num(new_size * item_size),
1683 new_ptr == lp->items ? " not" : "");
1684 }
1685 if (!new_ptr)
1686 out_of_memory("expand_item_list");
Although I imagine that output might be voluminous [but maybe not]?
Again, I don't have time to build test cases and reproduce this
carefully, every run is painful and long and slow. But I'd like to do the
responsible thing if someone can tell me what that is.
> p.s.: If I had to start over, I would have spent less time just deleting
> the data and recopying it, rather than trying to fixup the metadata and
Indeed, it's looking like fixing the metadata with rsync is an order
of magnitude slower, even as far as I've gotten. So maybe it's time to find
another method. I don't think fts(3) is optimized any better for large
hardlink farms, so I think maybe I need a homegrown solution? Ughhhh.
--jh...@mit.edu
John Hawkinson
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
