Re: [BackupPC-users] Help with monthly schedule configuration
Jamie- Another source of advice is the listserv archives. Assuming you're running BackupPC 4. If not, you'll want to upgrade! Our organization wants to keep at least one month of daily backups Here's a snippet of a post I did on this subject the last time someone asked about schedules. The values you'll want to check: $Conf{IncrKeepCnt} = 26; # This is the number of total 'unfilled' backups kept. $Conf{FillCycle} = 7;# This is how often a filled backup is kept (1 per week) which strongly influences the next setting $Conf{FullKeepCnt} = [ 4, 3, 0, ]; # This defines how many 'filled' backups are retained. The combination of filled and unfilled backups result in ~32 days of daily backups plus a couple of older ones just in case a user needs a file from more than a month ago. Remember, in BackupPC 4, a "filled" backup is kinda equivalent to a 'full' backup for the purposes of "FullKeepCnt", which should be renamed "FilledKeepCnt". Missing from that post was: $Conf{FullPeriod} = 90.97 This defines the time between 'full' backups where a full checksum is done against the source. But, as it turns out, my schedule above deletes the last 'full' backup after about 70 days, and BackupPC ends up taking a full immediately after that, so the "FullPeriod" never ends up being triggered. If my "FullKeepCnt" looked like [4, 3, 1, ]then my FullPeriod would trigger before the last full backup was aged out. The Filled vs. Full backups are a bit confusing. -- Ray Frush On Wed, Nov 15, 2017 at 2:59 PM, Jamie Burchellwrote: > Hi! > > > > Hoping someone can give me that “ah ha!” moment that I’m so desperately > craving after pouring over the documentation, mailing lists and various > forum posts. > > > > I want to move from BackupPC’s default schedule to keeping ~1 month’s > worth of backups, but I cannot fathom if I should: > > > > - Do a full backup every day and keep 30 of them > > - Do a full backup every week and keep 4 of them, with > incrementals in between > > - Do a full backup each month and keep 30 incrementals. > > > > BackupPC is so efficient with storage and transferring only what is needed > between backups that I don’t understand the difference between the three > approaches. All backups can be browsed like full backups, BackupPC only > ever transfers files it doesn’t have, all storage is deduplicated and rsync > can detect changes, new files and deletions, so why does it matter? I am > using rsync (over SSH), network speed and reliability is good and the > drives are all SSD. > > > > The settings I currently have are: > > > > FullPeriod 6.97 > > FullKeepCnt 4 > > IncrPeriod 0.97 > > IncrKeepCnt 24 > > > > I **think** this will give me 4 full backups with incrementals in > between, but I think I could have equally have gone with: > > > > FullPeriod 30 > > FullKeepCnt 1 > > IncrPeriod 0.97 > > IncrKeepCnt 29 > > > > I don’t understand what is meant by a “filled backup” either. > > > > Thanks for any clarity/help in advance! > > > > Regards > > Jamie > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > -- Time flies like an arrow, but fruit flies like a banana. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] error in rsync protocol data stream (code 12) (Restoring)
On Wed, 15 Nov 2017 23:29:58 - Jamie Burchellwrote: > Because you'll seldom find any good advice that advocates doing > anything as root. You misread it. Doing stuffs as root when it can be done by a regular user, or a sudo user _can_ be a risk (although I don't know any real admin that hasn't at least one or two consoles opened as root.) Everybody with a large practice of Linux has been goofing at least one time - I made a rm -r * as root on the root of the master disk (thanks BPC !), but you can also do tough shit as a user (such as rm -r ~ /dir watch the space between ~ and /dir, meaning you're removing your whole $HOME + /dir !) And in the particular scheme of BPC, where's the risk? root does launch rsync and recover files or parts of files into a controlled, so what? This isn't a _console_ risky line command order, this is part of a known automatic process ! In short: being root and (especially) removing directories is bad, on the other hand, using root as part of a controlled process doesn't mean that you'll be hacked or whatever - furthermore, doing some stuffs as root is compulsory for some maintenance work. Rule of thumb: don't get creative with a well known (and described) process. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] error in rsync protocol data stream (code 12) (Restoring)
Because you'll seldom find any good advice that advocates doing anything as root. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] error in rsync protocol data stream (code 12) (Restoring)
On Wed, 15 Nov 2017 22:48:01 - Jamie Burchellwrote: > I followed the instructions to make a restricted backuppc user on > client machines with limited sudo permission thus: > backuppc ALL=NOPASSWD: /usr/bin/rsync --server --sender * Why on earth did you use that instead of let it to root !? in which case, restoration by a user doesn't cause any problem. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] error in rsync protocol data stream (code 12) (Restoring)
Hi! I followed the instructions to make a restricted backuppc user on client machines with limited sudo permission thus: backuppc ALL=NOPASSWD: /usr/bin/rsync --server --sender * This works fine for backing up, but I just discovered I can no longer restore directly, as I’m getting the following error: Restore failed on X (rsync error: error in rsync protocol data stream (code 12) at io.c(629) [sender=3.0.9.8]) I just logged in to the client, and sure enough have this in the secure logs: Nov 15 22:39:54 x sudo: backuppc : command not allowed ; TTY=unknown ; PWD=/home/backuppc ; USER=root ; COMMAND=/usr/bin/rsync --server -slHogDtprRe.iLsf I’m assuming this is because the “--sender" parameter isn’t used here. What’s the best way to fix this? Should I simply remove “--sender" from the sudoers config and should this be added to the documentation? TIA Jamie -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Help with monthly schedule configuration
On Wed, 15 Nov 2017 22:52:26 - Jamie Burchellwrote: > I've gone with the schedule that looks correct for now and will see > what happens! Terrible things, you'll lose confidence in closed source softwares (hence, in m$ "products"), your HD will fill with many little gremlins, and finally you'll end with usable backups on an almost full month ! As I said, terrible !! -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Help with monthly schedule configuration
Thanks for the info. I can assure you I've read the docs, many times :) I'm clearly just a bit of a slow learner... I've gone with the schedule that looks correct for now and will see what happens! -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Help with monthly schedule configuration
On Wed, 15 Nov 2017 21:59:28 - Jamie Burchellwrote: > Hi! Ho¡ > Hoping someone can give me that “ah ha!” moment that I’m so desperately > craving after pouring over the documentation, mailing lists and various > forum posts. Jiff chops a “ah ha!” moment from the (near) field, cooks it and => Jamie Burchell a hot “ah ha!” moment. > I want to move from BackupPC’s default schedule to keeping ~1 month’s > worth of backups, but I cannot fathom if I should: > - Do a full backup every day and keep 30 of them > - Do a full backup every week and keep 4 of them, with > incrementals in between > - Do a full backup each month and keep 30 incrementals. For what it's worth (I'm not a BPC specialist, just a doc reader), I keep several filled incrementals each day (~a month), plus some fulls every sunday (5, with the last of them a real SOS as it is extra-old.) This way, as incrementals looks complete (and fulls looks… full;), I just use the last backup, pick what I need and restore it when needed. > BackupPC is so efficient with storage and transferring only what is > needed between backups that I don’t understand the difference between > the three approaches. IIRC, fulls are unconditionally full backups, meaning they do not care about whatever took place formerly, meaning you either take disk place for nothing if you do one/day (links are cheap but not free in term of HD place.) > All backups can be browsed like full backups, Yep, but only if you said so, part of BPC black magic. > BackupPC only ever transfers files it doesn’t have, all storage is > deduplicated and rsync can detect changes, new files and deletions, so > why does it matter? All this takes time when an incremental take usually (much) less time (NB: we're NOT talking about very busy and large databases that have a short rotation of the whole rows here, just regular data.) > FullPeriod 6.97 > FullKeepCnt 4 > IncrPeriod 0.97 > IncrKeepCnt 24 > I **think** this will give me 4 full backups with incrementals in > between, but I think I could have equally have gone with: Looks correct. > FullPeriod 30 > FullKeepCnt 1 > IncrPeriod 0.97 > IncrKeepCnt 29 No, as it will keep more fulls than necessary. > I don’t understand what is meant by a “filled backup” either. Reading the doc helps, a lot. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Help with monthly schedule configuration
Hi! Hoping someone can give me that “ah ha!” moment that I’m so desperately craving after pouring over the documentation, mailing lists and various forum posts. I want to move from BackupPC’s default schedule to keeping ~1 month’s worth of backups, but I cannot fathom if I should: - Do a full backup every day and keep 30 of them - Do a full backup every week and keep 4 of them, with incrementals in between - Do a full backup each month and keep 30 incrementals. BackupPC is so efficient with storage and transferring only what is needed between backups that I don’t understand the difference between the three approaches. All backups can be browsed like full backups, BackupPC only ever transfers files it doesn’t have, all storage is deduplicated and rsync can detect changes, new files and deletions, so why does it matter? I am using rsync (over SSH), network speed and reliability is good and the drives are all SSD. The settings I currently have are: FullPeriod 6.97 FullKeepCnt 4 IncrPeriod 0.97 IncrKeepCnt 24 I **think** this will give me 4 full backups with incrementals in between, but I think I could have equally have gone with: FullPeriod 30 FullKeepCnt 1 IncrPeriod 0.97 IncrKeepCnt 29 I don’t understand what is meant by a “filled backup” either. Thanks for any clarity/help in advance! Regards Jamie -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/