Sean Greenslade posted on Wed, 12 Oct 2016 18:29:55 -0400 as excerpted:
> Hi, all. I have a question about a backup plan I have involving
> send/receive. As far as I can tell, there's no way to to resume a send
> that has been interrupted. In this case, my interruption comes from an
> overbearing firewall that doesn't like long-lived connections. I'm
> trying to do the initial (non-incremental) sync of the first snapshot
> from my main server to my backup endpoint. The snapshot is ~900 GiB, and
> the internet link is 25 Mbps, so this'll be going for quite a long time.
> What I would like to do is "fake" the first snapshot transfer by
> rsync-ing the files over. So my question is this: if I rsync a subvolume
> (with the -a option to make all file times, permissions, ownerships,
> etc. the same),
> is that good enough to then be used as a parent for future incremental
I see the specific questions have been answered, and alternatives
explored in one direction, but I've another alternative, in a different
direction, to suggest.
First a disclaimer. I'm a btrfs user/sysadmin and regular on the list,
but I'm not a dev, and my own use-case doesn't involve send/receive, so
what I know regarding send/receive is from the list and manpages, not
personal experience. With that in mind...
It's worth noting that send/receive are subvolume-specific -- a send
won't continue down into a subvolume.
Also note that in addition to -p/parent, there's -s/clone-src. The
latter is more flexible than the super-strict parent option, at the
expense of a fatter send-stream as additional metadata is sent that
specifies which clone the instructions are relative to.
It should be possible to use the combination of these two facts to split
and recombine your send stream in a firewall-timeout-friendly manner, as
long as no individual files are so big that sending an individual file
exceeds the timeout.
1) Start by taking a read-only snapshot of your intended source
subvolume, so you have an unchanging reference.
2) Take multiple writable snapshots of it, and selectively delete subdirs
(and files if necessary) from each writable snapshot, trimming each one
to a size that should pass the firewall without interruption, so that the
combination of all these smaller subvolumes contains the content of the
single larger one.
3) Take read-only snapshots of each of these smaller snapshots, suitable
4) Do a non-incremental send of each of these smaller snapshots to the
If it's practical to keep the subvolume divisions, you can simply split
the working tree into subvolumes and send those individually instead of
doing the snapshot splitting above, in which case you can then use -p/
parent on each as you were trying to do on the original, and you can stop
If you need/prefer the single subvolume, continue...
5) Do an incremental send of the original full snapshot, using multiple
-c <src> options to list each of the smaller snapshots. Since all the
data has already been transferred in the smaller snapshot sends, this
send should be all metadata, no actual data. It'll simply be combining
the individual reference subvolumes into a single larger subvolume once
6) Once you have the single larger subvolume on the receive side, you can
delete the smaller snapshots as you now have a copy of the larger
subvolume on each side to do further incremental sends of the working
7) I believe the first incremental send of the full working copy against
the original larger snapshot will still have to use -c, while incremental
sends based on that first one will be able to use the stricter but
slimmer send-stream -p, with each one then using the previous one as the
parent. However, I'm not sure on that. It may be that you have to
continue using the fatter send-stream -c each time.
Again, I don't have send/receive experience of my own, so hopefully
someone who does can reply either confirming that this should work and
whether or not -p can be used after the initial setup, or explaining why
the idea won't work, but at this point based on my own understanding, it
seems like it should be perfectly workable to me. =:^)
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html