On 2014-07-24 19:56, Allan Jude wrote:
or better yet:
ssh r...@tbh.lerctr.org "zfs send ..." | mbuffer -m 16M | zfs recv ...
(The misc/mbuffer compensates for bursty zfs reads and writes.
 A note to myself: I should suggest to Allan to add mbuffer
 in a pipe as used in sysutils/zxfer, instead of patching zxfer
 for our local use :)

zxfer can already do this, with the -D option
I actually use misc/clpbar and get a progress bar as well

-D 'bar -s %%size%% -bl 1m -bs 128m'

or in your case: -D 'mbuffer -m 16M'

Thank you Allan! It shows it's been a while since the last time
I inspected the guts of zxfer.

2014-07-25 06:43 Larry Rosenman wrote:
Ok, I did the mbuffer trick, and the SEND side is where the memory issue is:
borg.lerctr.org /home/ler/bin $ tail zfs-send.log
23:28:12   15.7G   zroot/home@2014-07-24_22:56
23:28:13   15.7G   zroot/home@2014-07-24_22:56
23:28:14   15.7G   zroot/home@2014-07-24_22:56
23:28:15   15.7G   zroot/home@2014-07-24_22:56
23:28:16   15.7G   zroot/home@2014-07-24_22:56
23:28:17   15.7G   zroot/home@2014-07-24_22:56
23:28:18   15.7G   zroot/home@2014-07-24_22:56
23:28:19   15.7G   zroot/home@2014-07-24_22:56
23:28:20   15.8G   zroot/home@2014-07-24_22:56
Write failed: Cannot allocate memory
borg.lerctr.org /home/ler/bin $

Good information. So the "Write failed: Cannot allocate memory"
on the send side is what is causing the "invalid backup stream"
on the receiving side.

borg.lerctr.org /home/ler/bin $ tail zfs-recv.log
cannot receive new filesystem stream: invalid backup stream
borg.lerctr.org /home/ler/bin $

borg.lerctr.org /home/ler/bin $  cat backup-TBH-ZFS-initial.sh
DATE=`date "+%Y-%m-%d_%H:%M"`
#DATE2=`date -v "-1d" "+%Y-%m-%d"`
# snap the source
ssh r...@tbh.lerctr.org zfs snapshot -r zroot@${DATE}
# zfs copy the source to here.
ssh r...@tbh.lerctr.org 2>zfs-send.log "zfs send -v -R zroot@${DATE} " | \
     mbuffer -m 16M 2>mbuffer.log | \
     zfs recv -F -u -v -d zroot/backups/TBH3 2>zfs-recv.log
# make sure we NEVER allow the backup stuff to automount.
/sbin/zfs list -H -t filesystem -r zroot/backups/TBH3| \
    awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh
borg.lerctr.org /home/ler/bin $

borg.lerctr.org /home/ler/bin $ ssh tbh vmstat -z
Where to now?

Don't know, I'd guess some network-related memory limit is being hit
on the sending site.

Why not try to decouple the 'zfs send' from a network copy and ssh:
Login to a remote side, do a 'zfs send' to some local temporary file
there, then feed that file to ssh and send it over to a home host,
where it can be piped into some simple program like md5 or 'wc -c'
instead of a zfs recv.

freebsd-current@freebsd.org mailing list
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Reply via email to