Re: [BackupPC-users] Help with monthly schedule configuration

[Ray Frush]

The incremental period of 0.97 results in a daily backup, so that's
probably what you want to keep.


My schedule ends up giving you something like this:   ~32 daily backups + a
couple of older ones just in case you need an older file.

Backup# Type Filled Level Start Date Duration/mins Age/days
0 full yes 0 2017-09-20 18:00 0.3 56.9
14 incr yes 1 2017-10-04 19:00 0.1 42.9
26 incr no 1 2017-10-16 19:00 0.1 30.9
27 incr no 1 2017-10-17 19:00 0.1 29.9
28 incr yes 1 2017-10-18 19:00 0.1 28.9
29 incr no 1 2017-10-19 19:00 0.1 27.9
30 incr no 1 2017-10-20 19:00 0.1 26.9
31 incr no 1 2017-10-21 19:00 0.1 25.9
32 incr no 1 2017-10-22 19:00 0.1 24.9
33 incr no 1 2017-10-23 19:00 0.1 23.9
34 incr no 1 2017-10-24 19:00 0.1 22.9
35 incr yes 1 2017-10-25 22:00 0.1 21.8
36 incr no 1 2017-10-26 22:00 0.1 20.8
37 incr no 1 2017-10-27 22:00 0.1 19.8
38 incr no 1 2017-10-28 22:00 0.1 18.8
39 incr no 1 2017-10-29 22:00 0.1 17.8
40 incr no 1 2017-10-30 22:00 0.1 16.8
41 incr no 1 2017-10-31 22:00 0.1 15.8
42 incr yes 1 2017-11-01 22:00 0.1 14.8
43 incr no 1 2017-11-02 22:00 0.1 13.8
44 incr no 1 2017-11-03 22:00 0.1 12.8
45 incr no 1 2017-11-04 22:00 0.1 11.8
46 incr no 1 2017-11-05 21:00 0.1 10.8
47 incr no 1 2017-11-06 21:00 0.1 9.8
48 incr no 1 2017-11-07 21:00 0.1 8.8
49 incr yes 1 2017-11-08 21:00 0.2 7.8
50 incr no 1 2017-11-09 21:00 0.1 6.8
51 incr no 1 2017-11-10 21:00 0.1 5.8
52 incr no 1 2017-11-11 21:00 0.1 4.8
53 incr no 1 2017-11-12 21:00 0.1 3.8
54 incr no 1 2017-11-13 21:00 0.1 2.8
55 incr no 1 2017-11-14 21:00 0.1 1.8
56 incr yes 1 2017-11-15 21:00 0.1 0.8

On Thu, Nov 16, 2017 at 2:16 AM, Jamie Burchell  wrote:

> Thanks Ray
>
>
>
> I’ll be honest, I still don’t understand those settings after re-reading
> several times.
>
>
>
> What should the incremental period be here 0.97
>
>
>
> I’m also only interested in one month’s worth, so can that schedule be
> simplified?
>
>
>
> Jamie
>
>
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
>


-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Help with monthly schedule configuration

[Gerald Brandt]

On 2017-11-16 03:16 AM, Jamie Burchell wrote:


Thanks Ray

I’ll be honest, I still don’t understand those settings after 
re-reading several times.


What should the incremental period be here 0.97

I’m also only interested in one month’s worth, so can that schedule be 
simplified?


Jamie



Hi,

I  have:

Full Backups

FullPeriod 
 
	
FullKeepCnt 
 
	
FullKeepCntMin 
 
	
FullAgeMax 
 
	

Incremental Backups
IncrPeriod 
 
	
IncrKeepCnt 
 
	
IncrKeepCntMin 
 
	
IncrAgeMax 
 
	
IncrLevels 
 
	
IncrFill 
 
	



And then a few scripts that maintain what I want. Attached.

This gives me full backups every Friday and on the last day of the 
month, and keeps backups for as long as I want (specified in 
prune_backups.sh)


Gerald


Gerald


archive_backups.sh
Description: application/shellscript
#!/bin/bash

# force all hosts in the host file do have a full backup on Friday
# unless that Friday is the last workday of the month
if [ ! $(cal | tr -s '[:blank:]' '\n' |tail -1) = $(date +%d) ]
then
  for HOST in $(cat /etc/backuppc/hosts | grep ^[A-Za-z0-9] | grep -v host | 
grep -v archive | awk '{print $1}')
  do
/usr/share/backuppc/bin/BackupPC_serverMesg backup ${HOST} ${HOST} backuppc 
1
#   echo ${HOST}
  done
fi
#!/bin/bash

# force all hosts in the host file do have a full backup on last day of month
TODAY=`/bin/date +%d`
TOMORROW=`/bin/date +%d -d "1 day"`

#if [ $(cal | tr -s '[:blank:]' '\n' |tail -1) = $(date +%d) ]
if [ $TOMORROW -lt $TODAY ]
then
  for HOST in $(cat /etc/backuppc/hosts | grep ^[A-Za-z0-9] | grep -v host | 
grep -v archive | awk '{print $1}')
  do
/usr/share/backuppc/bin/BackupPC_serverMesg backup ${HOST} ${HOST} backuppc 
1
#   echo ${HOST}
  done
fi


prune_backups.sh
Description: application/shellscript
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Help with monthly schedule configuration

[Jamie Burchell]

Thanks Ray



I’ll be honest, I still don’t understand those settings after re-reading
several times.



What should the incremental period be here 0.97



I’m also only interested in one month’s worth, so can that schedule be
simplified?



Jamie
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Help with monthly schedule configuration

[Ray Frush]

Jamie-

Another source of advice is the listserv archives.   Assuming you're
running BackupPC 4.  If not, you'll want to upgrade!

Our organization wants to keep at least one month of daily backups

Here's a snippet of a post I did on this subject the last time someone
asked about schedules.

The values you'll want to check:
$Conf{IncrKeepCnt} = 26;  # This is the number of total 'unfilled'
backups kept.

$Conf{FillCycle} = 7;# This is how often a filled backup is kept (1 per
week) which strongly influences the next setting

$Conf{FullKeepCnt} = [  4,  3,  0, ];  # This defines how many 'filled'
backups are retained.

The combination of filled and unfilled backups result in ~32 days of daily
backups plus a couple of older ones just in case a user needs a file from
more than a month ago.



Remember, in BackupPC 4,  a "filled" backup is kinda equivalent to a 'full'
backup for the purposes of "FullKeepCnt", which should be renamed
"FilledKeepCnt".

Missing from that post was:
  $Conf{FullPeriod} = 90.97

This defines the time between 'full' backups where a full checksum is done
against the source.

But, as it turns out, my schedule above deletes the last 'full' backup
after about 70 days, and BackupPC ends up taking a full immediately after
that, so the "FullPeriod" never ends up being triggered.
If my "FullKeepCnt" looked like [4, 3, 1, ]then my FullPeriod would
trigger before the last full backup was aged out.  The Filled vs. Full
backups are a bit confusing.


--
Ray Frush


On Wed, Nov 15, 2017 at 2:59 PM, Jamie Burchell  wrote:

> Hi!
>
>
>
> Hoping someone can give me that “ah ha!” moment that I’m so desperately
> craving after pouring over the documentation, mailing lists and various
> forum posts.
>
>
>
> I want to move from BackupPC’s default schedule to keeping ~1 month’s
> worth of backups, but I cannot fathom if I should:
>
>
>
> -  Do a full backup every day and keep 30 of them
>
> -  Do a full backup every week and keep 4 of them, with
> incrementals in between
>
> -  Do a full backup each month and keep 30 incrementals.
>
>
>
> BackupPC is so efficient with storage and transferring only what is needed
> between backups that I don’t understand the difference between the three
> approaches. All backups can be browsed like full backups, BackupPC only
> ever transfers files it doesn’t have, all storage is deduplicated and rsync
> can detect changes, new files and deletions, so why does it matter? I am
> using rsync (over SSH), network speed and reliability is good and the
> drives are all SSD.
>
>
>
> The settings I currently have are:
>
>
>
> FullPeriod 6.97
>
> FullKeepCnt 4
>
> IncrPeriod 0.97
>
> IncrKeepCnt 24
>
>
>
> I **think** this will give me 4 full backups with incrementals in
> between, but I think I could have equally have gone with:
>
>
>
> FullPeriod 30
>
> FullKeepCnt 1
>
> IncrPeriod 0.97
>
> IncrKeepCnt 29
>
>
>
> I don’t understand what is meant by a “filled backup” either.
>
>
>
> Thanks for any clarity/help in advance!
>
>
>
> Regards
>
> Jamie
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
>


-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Help with monthly schedule configuration

[Bzzzz]

On Wed, 15 Nov 2017 22:52:26 -
Jamie Burchell  wrote:

> I've gone with the schedule that looks correct for now and will see
> what happens!

Terrible things, you'll lose confidence in closed source softwares
(hence, in m$ "products"), your HD will fill with many little gremlins, 
and finally you'll end with usable backups on an almost full month !
As I said, terrible !!

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Help with monthly schedule configuration

[Jamie Burchell]

Thanks for the info. I can assure you I've read the docs, many times :) I'm
clearly just a bit of a slow learner...

I've gone with the schedule that looks correct for now and will see what
happens!

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Help with monthly schedule configuration

[Bzzzz]

On Wed, 15 Nov 2017 21:59:28 -
Jamie Burchell  wrote:

> Hi!

Ho¡

> Hoping someone can give me that “ah ha!” moment that I’m so desperately
> craving after pouring over the documentation, mailing lists and various
> forum posts.

Jiff chops a “ah ha!” moment from the (near) field,
cooks it and => Jamie Burchell a hot “ah ha!” moment.

> I want to move from BackupPC’s default schedule to keeping ~1 month’s
> worth of backups, but I cannot fathom if I should:
> -  Do a full backup every day and keep 30 of them
> -  Do a full backup every week and keep 4 of them, with
> incrementals in between
> -  Do a full backup each month and keep 30 incrementals.

For what it's worth (I'm not a BPC specialist, just a doc reader), I keep
several filled incrementals each day (~a month), plus some fulls every
sunday (5, with the last of them a real SOS as it is extra-old.)

This way, as incrementals looks complete (and fulls looks… full;),
I just use the last backup, pick what I need and restore it when needed.

> BackupPC is so efficient with storage and transferring only what is
> needed between backups that I don’t understand the difference between
> the three approaches.

IIRC, fulls are unconditionally full backups, meaning they do not care
about whatever took place formerly, meaning you either take disk place
for nothing if you do one/day (links are cheap but not free in term of
HD place.)

> All backups can be browsed like full backups,

Yep, but only if you said so, part of BPC black magic.

> BackupPC only ever transfers files it doesn’t have, all storage is
> deduplicated and rsync can detect changes, new files and deletions, so
> why does it matter?

All this takes time when an incremental take usually (much) less time 
(NB: we're NOT talking about very busy and large databases that have a
short rotation of the whole rows here, just regular data.)

> FullPeriod 6.97
> FullKeepCnt 4
> IncrPeriod 0.97
> IncrKeepCnt 24
> I **think** this will give me 4 full backups with incrementals in
> between, but I think I could have equally have gone with:

Looks correct.

> FullPeriod 30
> FullKeepCnt 1
> IncrPeriod 0.97
> IncrKeepCnt 29

No, as it will keep more fulls than necessary.

> I don’t understand what is meant by a “filled backup” either.

Reading the doc helps, a lot.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] Help with monthly schedule configuration

[Jamie Burchell]

Hi!



Hoping someone can give me that “ah ha!” moment that I’m so desperately
craving after pouring over the documentation, mailing lists and various
forum posts.



I want to move from BackupPC’s default schedule to keeping ~1 month’s worth
of backups, but I cannot fathom if I should:



-  Do a full backup every day and keep 30 of them

-  Do a full backup every week and keep 4 of them, with
incrementals in between

-  Do a full backup each month and keep 30 incrementals.



BackupPC is so efficient with storage and transferring only what is needed
between backups that I don’t understand the difference between the three
approaches. All backups can be browsed like full backups, BackupPC only
ever transfers files it doesn’t have, all storage is deduplicated and rsync
can detect changes, new files and deletions, so why does it matter? I am
using rsync (over SSH), network speed and reliability is good and the
drives are all SSD.



The settings I currently have are:



FullPeriod 6.97

FullKeepCnt 4

IncrPeriod 0.97

IncrKeepCnt 24



I **think** this will give me 4 full backups with incrementals in between,
but I think I could have equally have gone with:



FullPeriod 30

FullKeepCnt 1

IncrPeriod 0.97

IncrKeepCnt 29



I don’t understand what is meant by a “filled backup” either.



Thanks for any clarity/help in advance!



Regards

Jamie
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/