On 13/10/16 00:47, Sean Greenslade wrote:
> I may just end up doing that. Hugo's responce gave me some crazy ideas
> involving a custom build of split that waits for a command after each
> output file fills, which would of course require an equally weird build
> of cat that would stall the pipe indefinitely until all the files showed
> up. Driving the HDD over would probably be a little simpler. =P
I am sure it is, if that is an option. I had considered doing something
similar: doing an initial send to a big file on a spare disk, then
sending the disk to Amazon to import using their Import/Export Disk
service, then creating a server and a filesystem in AWS and doing a
btrfs receive from the imported file. The plan would then be to do
incremental sends over my home broadband line for subsequent backups.
>>> And while we're at it, what are the failure modes for incremental sends?
>>> Will it throw an error if the parents don't match, or will there just be
>>> silent failures?
>> Create a list of possibilities, create some test filesystems, try it.
> I may just do that, presuming I can find the spare time. Given that I'm
> building a backup solution around this tech, it would definitely bolster
> my confidence in it if I knew what its failure modes looked like.
That's a good idea. In the end I decided that relying on btrfs send for
my offsite cold storage backups was probably not a good idea. Btrfs is
great, and I heavily use snapshotting and send/receive locally. But
there is always a small nagging fear that a btrfs bug could introduce
some problem which can survive send/receive and mean the backup was
corrupted as well (or that an incremental won't load for some reason).
For that reason I decided to deliberately use a different technology for
backups. I now use dar to create backups and then upload the files to a
cloud cold storage service for safekeeping.
There are other reasons as well: encryption is easier to handle, cold
storage for files is cheaper than having disk images which need to be
online to load the incrementals, no need for a virtual server, handling
security for backups from different servers with different levels of
risk is easier, etc. There are also downsides: verifying the backups are
readable/restorable is harder, bandwidth usage is less efficient (dar
sends more data than btrfs send would as it is working at the file
level, not the extent level).
By the way, to test out your various failure modes I recommend creating
some small btrfs filesystems on loop devices -- just be careful to make
sure you create each one from scratch and do not copy disk images (so
that they all have unique UIDs).
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html