Buttersink 0.6.2 - sync over ssh
I've just released version 0.6.2 of buttersink. It now supports syncing snapshots between a local btrfs and a local or remote btrfs over ssh, as well as syncing with S3 buckets. The utility is on PyPi as buttersink, and the GitHub page is here: https://github.com/AmesCornish/buttersink. Thanks in advance for any feedback! -- Ames -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Receive directory not empty bug
Sending tree data directly to Filipe, due to size and confidentiality... Thanks! -- Ames -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Receive directory not empty bug
Hi, btrfs is consistently refusing to receive send of certain directories. It fails with the message Directory not empty for an internal directory on the receiving side. Any ideas on what this might be or how to troubleshoot it? Here's an example: btrfs-progs v3.19-rc2 root@ProLiant:/mnt/backup/bug# uname -a Linux ProLiant 3.18.7-031807-generic #201502110759 SMP Wed Feb 11 08:01:01 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux root@ProLiant:/mnt/backup/bug# btrfs send /bak/Portege/2014-06-01 | btrfs rec -v . At subvol /bak/Portege/2014-06-01 At subvol 2014-06-01 receiving subvol 2014-06-01 uuid=f80254c5-2f5a-b043-ad3e-e12287f7b7e8, stransid=105481 BTRFS_IOC_SET_RECEIVED_SUBVOL uuid=f80254c5-2f5a-b043-ad3e-e12287f7b7e8, stransid=105481 root@ProLiant:/mnt/backup/bug# btrfs send -p /bak/Portege/2014-06-01 /bak/Portege/2014-12-01_01-02-44 | btrfs rec - . At subvol /bak/Portege/2014-12-01_01-02-44 At snapshot 2014-12-01_01-02-44 receiving snapshot 2014-12-01_01-02-44 uuid=b277e2ab-f0af-1e46-a8a3-cd587fd64a53, ctransid=474567 parent_uuid=f80254c5-2f5a-b043-ad3e-e12287f7b7e8, parent_ctransid=105481 utimes rename .duplicity - o257-9667-0 mkdir o257-473948-0 rename o257-473948-0 - .duplicity utimes chown .duplicity - uid=0, gid=100 chmod .duplicity - mode=02775 utimes .duplicity rename .ecryptfs - o258-9667-0 mkdir o258-473948-0 rename o258-473948-0 - lost+found utimes chown lost+found - uid=0, gid=0 chmod lost+found - mode=0700 utimes lost+found rmdir lost+found mkdir o259-473948-0 rename o259-473948-0 - monitor ERROR: rename o259-473948-0 - monitor failed. Directory not empty -- Ames -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Buttersink 0.4.1 - like rsync for btrfs snapshots
I've just released version 0.4.1 of buttersink. As of this version, I consider it beta and out of the experimental phase. Buttersink will synchronize a set of read-only snapshots in a btrfs filesystem to an Amazon S3 bucket, and vice-versa. It intelligently picks parent snapshots to diff from and send, so that a minimal amount of data needs to be sent over the wire and stored in the backend. I've put a fair amount of effort into making buttersink efficient and reliable enough for backups. Currently implemented features include: - Transfers between a local btrfs filesystem and an Amazon S3 bucket - Automatically synchronizes a set of snapshots, or a single snapshot, transferring only needed differences - Intelligent selection of full and incremental transfers to minimize costs of transfer and storage, and to minimize risks from corruption of a difference - Smart heuristics based on S3 file sizes, btrfs quota information, and btrfs-tools internal snapshot parent identification (ruuid) - Resumable, checksummed multi-part uploads to S3 as a back-end - Robust handling of btrfs send and receive errors - Configurable verbosity and logging - Conveniently lists snapshots and sizes in either btrfs or S3 - Detects and (optionally) deletes failed partial transfers If there's interest, I'll extend it to sync to a remote btrfs filesystem as well. The utility is on PyPi as buttersink, and the GitHub page is here: https://github.com/AmesCornish/buttersink. Thanks in advance for any feedback! - Ames -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Announcement: buttersink - like rsync for btrfs snapshots
I've written a utility to help me with using btrfs send and receive for backups or other synchronization, and I'd love to get feedback on it. As of this release, buttersink will synchronize a set of read-only snapshots in a btrfs filesystem to an Amazon S3 bucket, and vice-versa. It intelligently picks parent snapshots to diff from, so that a minimal amount of data needs to be sent over the wire and stored in the backend. The utility is on PyPi as buttersink, and the GitHub page is here: https://github.com/AmesCornish/buttersink. Thanks in advance for any feedback! - Ames -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcement: buttersink - like rsync for btrfs snapshots
Jim, btrfs send reliability has been an issue, though I've been able to successfully use it for my backups. buttersink usually detects the errors and will either move the destination snapshot to mark it as partial/failed (for btrfs), or cancel and delete the partial upload (for S3). I've also found that it helps to wait a while (e.g. 30 seconds) after any volume deletes before trying the send/sync. I hope btrfs-progs will get more reliable, too. - Ames -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
