Re: [BackupPC-users] Scheduling advice

[Kris Lou]

>
>
> I don't really know the details of bpc4.  I think it always fills the
> most recent run and works backwards to clean up the old copies so you
> should have whatever files are still there even if some were somehow
> deleted.   Still, if your pool filesystem is totally corrupted all
> bets are off - like it would be with any system
>

That is my understanding as well -- the most recent (and otherwise
configured) backup is always filled, and the previous ones are
reverse-delta'd from that.

I think the problem here is that filled != full, which is a concept that
I'm still wrapping my head around.  Your scenario assumes that subsequent
incrementals are always built upon a full backup, which was the behavior of
v3.  This would then cause problems when the full is deleted.

But here's how I think v4 is handling backups:  Upon a new backup ...
* Copies/renames the previous (#N-1), most recent (which is always filled)
backup as #N
* Finds and transfers new files into #N
* Pushes OLD (aka original) files into #N-1

So, Backup #2 then only holds the files that were changed, with the changed
files + unchanged files in Backup #3.  If you remove Backup #2, then you've
only lost the old versions of files that were changed in #3.

Craig notes in a few places that _any_ backup can be deleted -- I'm
assuming that BackupPC_backupDelete accounts for filling Backup #2 if
Backup #3 is deleted.  But if the underlying filesystem itself is mucked up
...

At least, that's what I get from staring at the v4 docs for a while.

-Kris
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Les Mikesell]

On Thu, Aug 31, 2017 at 11:51 AM, Gandalf Corvotempesta
 wrote:
> >
> Yes, now it's clear.
> But my issue is not bandwidth but time. A longer backup will increase
> load on the
> host for more time.

Yes, you need to pick your tradeoff between knowing your backup copy
is exactly correct and the amount of work the host does.  But you'd
need to have a pretty good reason to change the default weekly full,
like your host not being able to function or a weekend not being long
enough to complete.

> So, a filled backup is only needed to prevent bit-rot or something
> similiar, right ?
> If my filesystem is ZFS, can I safely use a single filled backup for
> many months?
> There is no need to compare checksum in bpc, because ZFS already does this.

Some filesystems allow certain methods of writing that do not update
the directory timestamp.  If you also don't change the length, an
rsync incremental won't read that file or back up the change.And
personally, I always liked the idea of regularly reading files to
detect hardware issues as early as possible.

>> "Filled" backups don't take a lot more space, just more time to build
>> the directory structure.  If you are concerned about this, keep more
>> filled copies.  In any case the next run will copy in anything
>> missing.
>
> Yes, on the run everyhting missing is synced. But what about a restore?
> If "file1" was created in the filled backup (now missing) and
> untouched in the subsequent incremental backup
> (thus it was not transferred), loosing the filled backup means to
> loose the "file1" ?

I don't really know the details of bpc4.  I think it always fills the
most recent run and works backwards to clean up the old copies so you
should have whatever files are still there even if some were somehow
deleted.   Still, if your pool filesystem is totally corrupted all
bets are off - like it would be with any system.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Ray Frush]

On Thu, Aug 31, 2017 at 10:45 AM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:

>
> Ok but let's simulate a crash in your example:
>
> On day 2, before the incremental backup, the filled one (day0) is lost.
> Is backup made on day1 still available with "all" files or only with
> the latest 5GB added between day0 and day1 ?


If you went to disk, and manually deleted backup "0" , backup 1 would still
exist, with references to all the new files, but I cannot predict how
BackupPC would react to the missing 'filled' full backup, nor am I willing
to test that for you.  You would be in a really unusual, unexpected state,
which is why BackupPC documentation suggests that you have regular 'filled'
available.


Finally, don't confuse a 'filled' backup with a 'full' backup.  A full
backup is 'filled' by default, but an incremental backup can be filled (
every 7 days using default settings) to help BackupPC keep track of things
better without any additional data transfer or checksum time.   A 'full'
backup is used to prevent bit rot.  Pick a timeframe that makes sense in
your environment.  Quarterly works for us, but I considered every 6 months.


I would suggest that you try it out, and before you depend on it, try
breaking BackupPC in all the ways you're worried about and see how it
behaves.   A lot of what you're asking us results in speculative answers
because we haven't tried deliberately breaking  our backups in the ways you
suggest.


-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

2017-08-31 18:51 GMT+02:00 Gandalf Corvotempesta
:
> Yes, on the run everyhting missing is synced. But what about a restore?

*on the NEXT run

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

2017-08-31 18:44 GMT+02:00 Les Mikesell :
> With rsync xfers, only the changes are going to be transferred.  The
> difference in a backuppc full and incremental is that the incremental
> will use the rsync feature of comparing the timestamp and length of
> the files to quickly skip unchanged files, where a full run will do a
> full read of all files on the target host for a block checksum
> comparison with the old copy.   If you use checksum-caching, the
> backuppc side will store those on the 2nd full run and not have to
> uncompress and compare for the third and subsequent full runs -
> however the client side always does a full read so fulls take more
> time but not a lot more bandwidth.   Bpc3 required the old matching
> file to have been in the same location on the same host to avoid
> transferring again.  Bpc4 is supposed to be able to identify matching
> files out of the pool if they have been renamed or you already have a
> copy from another host.   So if that "new" 5 GB was copied from
> somewhere that was already backed up, you would not need to transfer
> anything again.

Yes, now it's clear.
But my issue is not bandwidth but time. A longer backup will increase
load on the
host for more time.

So, a filled backup is only needed to prevent bit-rot or something
similiar, right ?
If my filesystem is ZFS, can I safely use a single filled backup for
many months?
There is no need to compare checksum in bpc, because ZFS already does this.

> "Filled" backups don't take a lot more space, just more time to build
> the directory structure.  If you are concerned about this, keep more
> filled copies.  In any case the next run will copy in anything
> missing.

Yes, on the run everyhting missing is synced. But what about a restore?
If "file1" was created in the filled backup (now missing) and
untouched in the subsequent incremental backup
(thus it was not transferred), loosing the filled backup means to
loose the "file1" ?

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

2017-08-31 18:34 GMT+02:00 Ray Frush :
> I'll extend the example
>
> Day 0  :  Full backup 100GB transfered
> Day 1  : add 5GB ,   Incremental runs, 5GB transferred
> Day 2 : add 5GB  ,   Incremental, ~5GB transferred
> Day 3 ; add 5 GB,   Full runs.   ALL files check-summed.  Files with
> identical checksum are skipped, new/changed files transferred:  ~ 5GB
>You do incur extra TIME related to checksumming all the files, but
> you only transfer what's changed/new

Perfect.

> BackupPC requires a minimum of one filled backup.   If you gracefully delete
> a filled backup, I believe that BackupPC does intelligently fill the next
> backup.   If you were to delete the filled backup from the filesystem
> directly, It is simply missing, and BackupPC would have to build a restore
> tree referencing all of the available incrementals back to the most recent
> available filled backup.

Ok but let's simulate a crash in your example:

On day 2, before the incremental backup, the filled one (day0) is lost.
Is backup made on day1 still available with "all" files or only with
the latest 5GB added between day0 and day1 ?

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Les Mikesell]

On Thu, Aug 31, 2017 at 11:23 AM, Gandalf Corvotempesta
 wrote:
>
> So, with a "full" run, the second "full" is still seen as an
> incremental by rsync?
> Let's assume a 100GB host.
> bpc will backup that host for the first time. 100GB are transferred.
> The next day, only 5GB are added on that host. i'll force bpc to make
> a "full/filled" backup.
> How many GB are transferred, 105GB or only 5GB ?

With rsync xfers, only the changes are going to be transferred.  The
difference in a backuppc full and incremental is that the incremental
will use the rsync feature of comparing the timestamp and length of
the files to quickly skip unchanged files, where a full run will do a
full read of all files on the target host for a block checksum
comparison with the old copy.   If you use checksum-caching, the
backuppc side will store those on the 2nd full run and not have to
uncompress and compare for the third and subsequent full runs -
however the client side always does a full read so fulls take more
time but not a lot more bandwidth.   Bpc3 required the old matching
file to have been in the same location on the same host to avoid
transferring again.  Bpc4 is supposed to be able to identify matching
files out of the pool if they have been renamed or you already have a
copy from another host.   So if that "new" 5 GB was copied from
somewhere that was already backed up, you would not need to transfer
anything again.

>> With BackupPC 4.x  if you delete a 'filled' backup (why would you do that
>> anyway?)  It just makes BackupPC work harder since it has to rebuild
>> references back to an older filled backup, which cost time while doing a
>> restore.  So you'll only lose the single day that you delete.
>
> And what If I don't have any other filled backup but only incrementals
> made from the deleted "filled" ?

"Filled" backups don't take a lot more space, just more time to build
the directory structure.  If you are concerned about this, keep more
filled copies.  In any case the next run will copy in anything
missing.

-- 
  Les Mikesell
lesmikes...@gmail.com

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Ray Frush]

On Thu, Aug 31, 2017 at 10:23 AM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
>
>
> So, with a "full" run, the second "full" is still seen as an
> incremental by rsync?
> Let's assume a 100GB host.
> bpc will backup that host for the first time. 100GB are transferred.
> The next day, only 5GB are added on that host. i'll force bpc to make
> a "full/filled" backup.
> How many GB are transferred, 105GB or only 5GB ?
>

I'll extend the example

Day 0  :  Full backup 100GB transfered
Day 1  : add 5GB ,   Incremental runs, 5GB transferred
Day 2 : add 5GB  ,   Incremental, ~5GB transferred
Day 3 ; add 5 GB,   Full runs.   ALL files check-summed.  Files with
identical checksum are skipped, new/changed files transferred:  ~ 5GB
   You do incur extra TIME related to checksumming all the files, but
you only transfer what's changed/new




>
> > With BackupPC 4.x  if you delete a 'filled' backup (why would you do that
> > anyway?)  It just makes BackupPC work harder since it has to rebuild
> > references back to an older filled backup, which cost time while doing a
> > restore.  So you'll only lose the single day that you delete.
>
> And what If I don't have any other filled backup but only incrementals
> made from the deleted "filled" ?


BackupPC requires a minimum of one filled backup.   If you gracefully
delete a filled backup, I believe that BackupPC does intelligently fill the
next backup.   If you were to delete the filled backup from the filesystem
directly, It is simply missing, and BackupPC would have to build a restore
tree referencing all of the available incrementals back to the most recent
available filled backup.






> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

2017-08-31 17:54 GMT+02:00 Les Mikesell :
> I guess I'm missing why you would ever want to delete anything
> manually.  With bpc the actual files are going to be in the pool
> anyway and you almost certainly don't want to delete anything manually
> from there because you'd lose things that are pooled from other hosts.
>   In any case, though, if the next (rsync) run does not find an
> existing copy it should fill it back in.  Tar/smb backups would take a
> full run to recover since they only transfer new files by timestamp on
> incrementals.

Obviously i don't want to delete anything. It's just a safety reason.
I'll repeat my experience with Bacula: with Bacula you have Full,
Incrementals and Differentials.
Incrementals are made from the latest Full and they only store changed files.

When you have to restore an host, you need the latest Full and any
following Incrementals.
Now let's assume a scheduling with 1 full per month and 30 days of incrementals.
If you are at 29th of the month and, for whatever reason, you loose
the Full (unclean shutdown and so on)
you have lost the whole month of backup.

I've learnt this the hardway where an unclean shutdown in the server
room corrupted the Full file the last day of the month.
The same unclean shutdown crashed a server that I had to restore. The
broken server was the affected by the lost backup in bacula.
Thus, a single failure affected a month full of backups.

With rsnapshot this can't happen. If you loose a single file from any
backup point, that file is still available in any other backup point
(obviously, it must be present)
Any backup is totally unrelated to each other thanks to hardlinking.
You can make any kind of mess with any backup point and nothing will
break (except the single backup point that your are messign whith)

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

2017-08-31 17:32 GMT+02:00 Ray Frush :
> With BackupPC 4.x  we only take a 'full' every 90 days, and because we're
> using rsync, subsequent fulls aren't as painful as the first one.  We run
> the full to ensure that all checksums match to avoid silent data corruption
> on th storage

So, with a "full" run, the second "full" is still seen as an
incremental by rsync?
Let's assume a 100GB host.
bpc will backup that host for the first time. 100GB are transferred.
The next day, only 5GB are added on that host. i'll force bpc to make
a "full/filled" backup.
How many GB are transferred, 105GB or only 5GB ?

> With BackupPC 4.x  if you delete a 'filled' backup (why would you do that
> anyway?)  It just makes BackupPC work harder since it has to rebuild
> references back to an older filled backup, which cost time while doing a
> restore.  So you'll only lose the single day that you delete.

And what If I don't have any other filled backup but only incrementals
made from the deleted "filled" ?

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Les Mikesell]

On Thu, Aug 31, 2017 at 10:24 AM, Gandalf Corvotempesta
 wrote:
>
> I would like to use BPC (i've used v3 many years ago with success
> ,tried v4 last year and it was a total mess due to a bug now fixed)
> but the ability to delete (brutally, from command line, not from BPC)
> and backup point is mandatory for me.

I guess I'm missing why you would ever want to delete anything
manually.  With bpc the actual files are going to be in the pool
anyway and you almost certainly don't want to delete anything manually
from there because you'd lose things that are pooled from other hosts.
  In any case, though, if the next (rsync) run does not find an
existing copy it should fill it back in.  Tar/smb backups would take a
full run to recover since they only transfer new files by timestamp on
incrementals.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Ray Frush]

On Thu, Aug 31, 2017 at 9:16 AM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:

> 2017-08-31 16:33 GMT+02:00 Ray Frush :
>
> Thanks for the reply.
> In this case, you are making some full backups.
> I don't want to run any full backup except for the first one, like
> with rsnapshot.
> With rsnapshot, only incrementals are made. I don't want to run full
> backups because
> I have some very very huge servers that will took 2 or 3 days to
> transfer everything, but only
> a couple of hours to transfer the changed files during an incremental run.


With BackupPC 4.x  we only take a 'full' every 90 days, and because we're
using rsync, subsequent fulls aren't as painful as the first one.  We run
the full to ensure that all checksums match to avoid silent data corruption
on th storage



>
> So, what happens if I delete the filled backup ? I'll only loose that
> single backup point or even some subsequent incrementals are lost
> because some files was located to the "filled" backup ?
>

With BackupPC 4.x  if you delete a 'filled' backup (why would you do that
anyway?)  It just makes BackupPC work harder since it has to rebuild
references back to an older filled backup, which cost time while doing a
restore.  So you'll only lose the single day that you delete.


>
> rsnapshot make uses of hardlinks, thus, the only way to loose a file
> is to loose all hardlinks pointing to that file.
> on the first run, all files are transfered. On following runs
> (incrementals) only changed files are transfered, everything else is
> hardlinked to the first backup. If you loose the "first" backup, the
> hardlink is still resolved.
>
> With BPC is the same ?
>
>
BackupPC 4.x does not use hardlinks extensively, which is a big
improvement.   BackupPC 4.x keeps a hash tree of unique files, and a system
of file reference 'pointers' to build the backup tree.





> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

2017-08-31 17:18 GMT+02:00 Les Mikesell :
> Also, note that backuppc's compression and pooling across host will
> likely at least double the history you can keep online unless your
> data is mostly unique and already compressed.

This is not an issue for me.
My biggest concern is how bpc handle a broken backup point. I had
multiple issues in the past
with bacula, where if you loose/corrupt the full backup, every other
following backup is gone.

rsnapshot solved this flawlessly, but doesn't offer any compression
feature and doesn't have any useful web interface.

I would like to use BPC (i've used v3 many years ago with success
,tried v4 last year and it was a total mess due to a bug now fixed)
but the ability to delete (brutally, from command line, not from BPC)
and backup point is mandatory for me.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Les Mikesell]

On Thu, Aug 31, 2017 at 9:33 AM, Ray Frush  wrote:
>
> To answer your second question:  BackupPC does a good job of managing the
> 'filled' (think 'full') backups if you decide to delete one.   I have found,
> that BackupPC is pretty good at self healing from issues.  We had a number
> of backups impacted by running out of inodes during a cycle.   While the
> files lost by the lack of Inodes cannot be recovered, BackupPC recovered
> gracefully on the next cycle after the file system was expanded.

Also, note that backuppc's compression and pooling across host will
likely at least double the history you can keep online unless your
data is mostly unique and already compressed.

-- 
   Les Mikesell
 lesmikesell@

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

2017-08-31 16:33 GMT+02:00 Ray Frush :
> BackupPC, is relatively easy to setup for a schedule like you propose.   We
> keep a 30 day backup history with a few extra weeks tacked on to get out to
> ~70 days, so the values below reflect our schedule:
>
> The values you'll want to check:
> $Conf{IncrKeepCnt} = 26;  # This is the number of total 'unfilled'
> backups kept.
>
> $Conf{FillCycle} = 7;# This is how often a filled backup is kept (1 per
> week) which strongly influences the next setting
>
> $Conf{FullKeepCnt} = [  4,  3,  0, ];  # This defines how many 'filled'
> backups are retained.
>
> The combination of filled and unfilled backups result in ~32 days of daily
> backups plus a couple of older ones just in case a user needs a file from
> more than a month ago.

Thanks for the reply.
In this case, you are making some full backups.
I don't want to run any full backup except for the first one, like
with rsnapshot.
With rsnapshot, only incrementals are made. I don't want to run full
backups because
I have some very very huge servers that will took 2 or 3 days to
transfer everything, but only
a couple of hours to transfer the changed files during an incremental run.

> To answer your second question:  BackupPC does a good job of managing the
> 'filled' (think 'full') backups if you decide to delete one.   I have found,
> that BackupPC is pretty good at self healing from issues.  We had a number
> of backups impacted by running out of inodes during a cycle.   While the
> files lost by the lack of Inodes cannot be recovered, BackupPC recovered
> gracefully on the next cycle after the file system was expanded.

So, what happens if I delete the filled backup ? I'll only loose that
single backup point or even some subsequent incrementals are lost
because some files was located to the "filled" backup ?

rsnapshot make uses of hardlinks, thus, the only way to loose a file
is to loose all hardlinks pointing to that file.
on the first run, all files are transfered. On following runs
(incrementals) only changed files are transfered, everything else is
hardlinked to the first backup. If you loose the "first" backup, the
hardlink is still resolved.

With BPC is the same ?

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Ray Frush]

Gandalf-

BackupPC, is relatively easy to setup for a schedule like you propose.   We
keep a 30 day backup history with a few extra weeks tacked on to get out to
~70 days, so the values below reflect our schedule:

The values you'll want to check:
$Conf{IncrKeepCnt} = 26;  # This is the number of total 'unfilled'
backups kept.

$Conf{FillCycle} = 7;# This is how often a filled backup is kept (1 per
week) which strongly influences the next setting

$Conf{FullKeepCnt} = [  4,  3,  0, ];  # This defines how many 'filled'
backups are retained.

The combination of filled and unfilled backups result in ~32 days of daily
backups plus a couple of older ones just in case a user needs a file from
more than a month ago.

To answer your second question:  BackupPC does a good job of managing the
'filled' (think 'full') backups if you decide to delete one.   I have
found, that BackupPC is pretty good at self healing from issues.  We had a
number of backups impacted by running out of inodes during a cycle.   While
the files lost by the lack of Inodes cannot be recovered, BackupPC
recovered gracefully on the next cycle after the file system was expanded.

--
Ray Frush



On Thu, Aug 31, 2017 at 4:39 AM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:

> Additionally, what happens if I delete/lost/break the full backups ?
> Any subsequent incremental backups will be broken or automatically the
> following incremental backup would become a "full" like with
> rsnapshot?
>
> 2017-08-30 21:54 GMT+02:00 Gandalf Corvotempesta
> :
> > Hi to all
> > Currently I use rsnapshot with success to backup about 20 hosts
> >
> > Our configuration is simple: every night I'll start 4 concurrent backups
> > keeping at least 10 days of old backups
> >
> > In this way, due to rsnapshot hardlinks , I'm able to restore any file
> up to
> > 10 days ago or to keep the backup chain consistent even by deleting
> multiple
> > backup trees at all.
> >
> > How can I get the same with bpc 4?
> > Last time I've tried I had difficult to understand filled/unfilled
> backups
> > and retention times
> >
> > Any help ? I would like to move to bpc due to it's deduplication and
> > compression feature but keeping the ability to destroy backup trees
> without
> > compromising the whole host backup is mandatory
> >
> > (In other words, with Bacula if you loose the full backup, you also loose
> > the whole backup chain, with rsnapshot there is no full, if you loose a
> > backup point, you'll only loose that backup point
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Scheduling advice

[Gandalf Corvotempesta]

Additionally, what happens if I delete/lost/break the full backups ?
Any subsequent incremental backups will be broken or automatically the
following incremental backup would become a "full" like with
rsnapshot?

2017-08-30 21:54 GMT+02:00 Gandalf Corvotempesta
:
> Hi to all
> Currently I use rsnapshot with success to backup about 20 hosts
>
> Our configuration is simple: every night I'll start 4 concurrent backups
> keeping at least 10 days of old backups
>
> In this way, due to rsnapshot hardlinks , I'm able to restore any file up to
> 10 days ago or to keep the backup chain consistent even by deleting multiple
> backup trees at all.
>
> How can I get the same with bpc 4?
> Last time I've tried I had difficult to understand filled/unfilled backups
> and retention times
>
> Any help ? I would like to move to bpc due to it's deduplication and
> compression feature but keeping the ability to destroy backup trees without
> compromising the whole host backup is mandatory
>
> (In other words, with Bacula if you loose the full backup, you also loose
> the whole backup chain, with rsnapshot there is no full, if you loose a
> backup point, you'll only loose that backup point

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/