Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-07 Thread Edward Ned Harvey (blu)
> From: discuss-bounces+blu=nedharvey@blu.org [mailto:discuss- > bounces+blu=nedharvey@blu.org] On Behalf Of Jerry Feldman > > > Btrfs and Linux LVM are > > the odd ducks here being the ones that lack a native mechanism for > > replicating snapshots to other media. > > > And this is the sp

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-07 Thread Tom Metro
Richard Pieri wrote: > rsnapshot is, as the name suggests, a snapshot system. It uses a > combination of GNU cp's hard link directory replication and rsync itself > to maintain time-based snapshots. It functions similarly to Apple's Time > Machine... [...] > There are two big drawbacks to rsnapshot

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-07 Thread Jerry Feldman
On 12/06/2013 01:55 PM, Richard Pieri wrote: > Jerry Feldman wrote: >> I'm not sure what it would use cp(1) for when backup up remotely. I > > It's not for remotely. Its for locally including over NFS. If I'm > remembering things properly at this point, "cp --link source target" > followed by an rs

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-07 Thread Jerry Feldman
On 12/05/2013 11:24 PM, Edward Ned Harvey (blu) wrote: >> From: Derek Atkins [mailto:warl...@mit.edu] >> Sent: Thursday, December 05, 2013 2:11 PM >> To: Edward Ned Harvey (blu) >> Cc: John Abreau; Jerry Feldman; BLU Discuss >> Subject: Re: [Discuss] rsnapshot vs.

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-06 Thread Jerry Feldman
On 12/05/2013 11:19 AM, Richard Pieri wrote: > John Abreau wrote: >> It occurs to me that I'm thinking of LVM snapshots, and I'm assuming >> btrfs >> is a filesystem that has LVM-like functionality baked in. > > That's correct. > > Btrfs snapshots, like LVM and ZFS and Advfs and XFS and AFS snapsho

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-06 Thread Richard Pieri
Jerry Feldman wrote: I'm not sure what it would use cp(1) for when backup up remotely. I It's not for remotely. Its for locally including over NFS. If I'm remembering things properly at this point, "cp --link source target" followed by an rsync update is much faster than "rsync --link-dest pr

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-06 Thread Jerry Feldman
On 12/05/2013 09:03 AM, Richard Pieri wrote: > Jerry Feldman wrote: >> It has used --link-dest as long as I have been using it. In the >> rsnapshot.conf file: > > Mine are the same, but there's still this early on: > >> # LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra >> feature

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread John Abreau
2013 2:11 PM > > To: Edward Ned Harvey (blu) > > Cc: John Abreau; Jerry Feldman; BLU Discuss > > Subject: Re: [Discuss] rsnapshot vs. rdiff-backup > > > > "Edward Ned Harvey (blu)" writes: > > > > > With my configuration, I get snapshot date

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Edward Ned Harvey (blu)
> From: Derek Atkins [mailto:warl...@mit.edu] > Sent: Thursday, December 05, 2013 2:11 PM > To: Edward Ned Harvey (blu) > Cc: John Abreau; Jerry Feldman; BLU Discuss > Subject: Re: [Discuss] rsnapshot vs. rdiff-backup > > "Edward Ned Harvey (blu)" writes: >

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Edward Ned Harvey (blu)
> From: discuss-bounces+blu=nedharvey@blu.org [mailto:discuss- > bounces+blu=nedharvey@blu.org] On Behalf Of Kent Borg > > Once the snapshot exists, it is there, self-consistent and unchanging, > it can be read at leisure to copy off to a backup. The volume doesn't > need to be unmounted

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Edward Ned Harvey (blu)
> From: John Abreau [mailto:abre...@gmail.com] > > A limitation of btrfs is that the snapshots are internal to the filesystem. > Kind > of makes it hard to put the snapshots on a separate machine. https://btrfs.wiki.kernel.org/index.php/Incremental_Backup > It occurs to me that I'm thinking of

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Derek Atkins
"Edward Ned Harvey (blu)" writes: > With my configuration, I get snapshot dates as follows: > > Nov 2 01:00 weekly.3/ > Nov 9 01:00 weekly.2/ > Nov 16 01:00 weekly.1/ > Nov 23 01:00 weekly.0/ Why is your weekly.0 more than a week out of date? I would've expected, based on the numbers, that yo

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Derek Atkins
Kent Borg writes: > A big consideration when looking at backups is to walk through the > restore scenario: how long will it take? Will that be good enough? > > I have used a home-brew backup scheme ping-ponged between two > different removable drives (one always disconnected, usually both are > d

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Richard Pieri
Kent Borg wrote: Ah, so you are not talking about just backing up your directories and files, you are talking about backing up the snapshot history, too? Not as such, no. I'm talking about making snapshots as read-only versions of a file system at arbitrary points in time and running the back

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Kent Borg
Ah, so you are not talking about just backing up your directories and files, you are talking about backing up the snapshot history, too? I feel an infinite regress coming on. I haven't played with btrfs in part because I feared that once I could make easy and cheap snapshots, I might actually

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Richard Pieri
Kent Borg wrote: What mechanisms do other systems have to make this easier? Some internals-to-internals data transport that is more efficient, opportunistic, etc., than a file interface? Advfs has vdump and vrestore. ZFS has send and receive, and the native Solaris tar and cpio retain ZFS fil

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Kent Borg
On 12/05/2013 11:19 AM, Richard Pieri wrote: Btrfs and Linux LVM are the odd ducks here being the ones that lack a native mechanism for replicating snapshots to other media. Once the snapshot exists, it is there, self-consistent and unchanging, it can be read at leisure to copy off to a backup

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Richard Pieri
John Abreau wrote: It occurs to me that I'm thinking of LVM snapshots, and I'm assuming btrfs is a filesystem that has LVM-like functionality baked in. That's correct. Btrfs snapshots, like LVM and ZFS and Advfs and XFS and AFS snapshots all reside on the same medium as the original. Btrfs an

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread John Abreau
It occurs to me that I'm thinking of LVM snapshots, and I'm assuming btrfs is a filesystem that has LVM-like functionality baked in. On Thu, Dec 5, 2013 at 11:00 AM, John Abreau wrote: > A limitation of btrfs is that the snapshots are internal to the filesystem. > Kind of makes it hard to put t

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread John Abreau
A limitation of btrfs is that the snapshots are internal to the filesystem. Kind of makes it hard to put the snapshots on a separate machine. On Thu, Dec 5, 2013 at 9:47 AM, Edward Ned Harvey (blu) wrote: > > From: discuss-bounces+blu=nedharvey@blu.org [mailto:discuss- > > bounces+blu=nedhar

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Edward Ned Harvey (blu)
> From: discuss-bounces+blu=nedharvey@blu.org [mailto:discuss- > bounces+blu=nedharvey@blu.org] On Behalf Of John Abreau > > I've never liked the way rsnapshot handles the additional levels of > snapshots. As I recall, the most recent weekly was several weeks old, the > most recent monthly

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Edward Ned Harvey (blu)
> From: discuss-bounces+blu=nedharvey@blu.org [mailto:discuss- > bounces+blu=nedharvey@blu.org] On Behalf Of Mike Small > > Does btrfs fit in here somewhere? Yes. When you create a btrfs or zfs snapshot, and send incremental to the destination system, that is perfectly analogous to rsna

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Richard Pieri
Jerry Feldman wrote: It has used --link-dest as long as I have been using it. In the rsnapshot.conf file: Mine are the same, but there's still this early on: # LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features. # EVERYONE ELSE: Leave "cmd_cp" commented out for compat

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Jerry Feldman
On 12/04/2013 11:37 AM, Richard Pieri wrote: > John Abreau wrot >> I've never liked the way rsnapshot handles the additional levels of >> snapshots. As I recall, the most recent weekly was several weeks old, >> the >> most recent monthly was always several months old, etc. > > This suggests to me

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Jerry Feldman
Since rsnapshot always grabs the oldest of the previous type (eg oldest hourly, daily, et. al), there is a very simple solution. Just set up a simple shell script to use the GNU cp(1) using the link option. On the first day of the month, simply schedule the script to link the proper hourly or dail

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-05 Thread Jerry Feldman
On 12/03/2013 07:12 PM, Richard Pieri wrote: > Jerry Feldman wrote: >> rsnapshot can run on BSD. It uses the rsync --link-dest feature. You >> generally have an hourly.0 directory. When rsnapshot runs it renames > > Really? That may have changed in the past few years. Used to be that > rsnapshot us

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-04 Thread John Abreau
I never used the hourly level, just the daily, weekly, monthly, and yearly. I believe the higher levels were each grabbing the oldest instance of the level immediately below it, rather than the newest instance. i wanted the higher levels to use the newest, so that, e.g.,, the monthly snapshot made

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-04 Thread Richard Pieri
John Abreau wrote: I've never liked the way rsnapshot handles the additional levels of snapshots. As I recall, the most recent weekly was several weeks old, the most recent monthly was always several months old, etc. This suggests to me that the scheduling is off. It could be due to incorrect

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread John Abreau
I've never liked the way rsnapshot handles the additional levels of snapshots. As I recall, the most recent weekly was several weeks old, the most recent monthly was always several months old, etc. Also, when I used rsnapshot to back up dozens of servers, I found that if only one portion of one s

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Richard Pieri
Jerry Feldman wrote: rsnapshot can run on BSD. It uses the rsync --link-dest feature. You generally have an hourly.0 directory. When rsnapshot runs it renames Really? That may have changed in the past few years. Used to be that rsnapshot used GNU cp's --link option to copy hourly.1 to hourly.0

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Jerry Feldman
On 12/03/2013 03:24 PM, Richard Pieri wrote: > Mike Small wrote: >> Does btrfs fit in here somewhere? I was using DragonFlyBSD for awhile, > > Not as such, no. File system and logical volume snapshots are not > backups. They're snapshots. > > >> Also, apparently rsync itself has a --link-dest featu

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Richard Pieri
Mike Small wrote: Does btrfs fit in here somewhere? I was using DragonFlyBSD for awhile, Not as such, no. File system and logical volume snapshots are not backups. They're snapshots. Also, apparently rsync itself has a --link-dest feature which maybe can be used to accomplish at least some

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Mike Small
Richard Pieri writes: > I've been using rsnapshot for several years now and I'm reasonably > familiar with it. It was recently suggested to me to use rdiff-backup > to copy files to a FAT32 file system because it is aware of FAT32 and > exFAT file name restrictions. Since then I've been experimen

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Jerry Feldman
On 12/03/2013 10:12 AM, Kent Borg wrote: > A big consideration when looking at backups is to walk through the > restore scenario: how long will it take? Will that be good enough? > > I have used a home-brew backup scheme ping-ponged between two > different removable drives (one always disconnected,

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Jerry Feldman
On 12/03/2013 09:54 AM, Richard Pieri wrote: > Jerry Feldman wrote: >> scheduling in cron. At work we had a WD MyBook which is a very, very >> slow device. Our first backup took days. (The WD was connected to the >> same switch as our NAS). I also had to schedule the rsnapshot backups >> along with

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Kent Borg
A big consideration when looking at backups is to walk through the restore scenario: how long will it take? Will that be good enough? I have used a home-brew backup scheme ping-ponged between two different removable drives (one always disconnected, usually both are disconnected and at least on

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Richard Pieri
Jerry Feldman wrote: scheduling in cron. At work we had a WD MyBook which is a very, very slow device. Our first backup took days. (The WD was connected to the same switch as our NAS). I also had to schedule the rsnapshot backups along with an offsite backup to our New York Office. The one thing

Re: [Discuss] rsnapshot vs. rdiff-backup

2013-12-03 Thread Jerry Feldman
Thanks for the descriptions Rich, While I did not think that configuring rsnapshot was tedious, and it is reasonably well documented. One issue I had, that you mention, is the scheduling in cron. At work we had a WD MyBook which is a very, very slow device. Our first backup took days. (The WD was c

[Discuss] rsnapshot vs. rdiff-backup

2013-12-02 Thread Richard Pieri
I've been using rsnapshot for several years now and I'm reasonably familiar with it. It was recently suggested to me to use rdiff-backup to copy files to a FAT32 file system because it is aware of FAT32 and exFAT file name restrictions. Since then I've been experimenting with rdiff-backup. Here