Re: [BackupPC-users] Outdated Files in Backup?
Thanks again!
Again, I do not know why this changed. Removed the entry on the host and
let the default settings settle in.
Thanks!
/KNEBB
Am 26.11.23 um 13:51 schrieb jbk:
On 11/26/23 05:30, Christian Völker wrote:
> Hi, > > thanks for pointins this out. I do not remember having set this >
manually on all hosts. It was set globally anyways so I removed these
> entries from all clients. > > And I have only a username for this
single host (as it is MacOS and > does not know "root" as the others
do). > > Meanwhile I have removed the host an created it from scratch.
Started > a backupc and now everything seems to work fine. I do not
know how > this has happened. > > Thanks! > > /KNEBB > > > Am 24.11.23
um 17:05 schrieb jbk: >> On 11/14/23 14:09, Christian Völker wrote:
>>> Hi, >>> >>> I have BackupPC 4.x running on a Debian system. Works
like a >>> champ since years now. >>> >>> One of my clients is an
Apple Mac with current up-to-date MacOS. >>> rsync is installed: >>>
>>> username@macos Documents % rsync --version rsync version 2.6.9 >>>
protocol version 29 Copyright (C) 1996-2006 by Andrew Tridgell, >>>
Wayne Davison, and others. So far it works fine. It does the >>>
backupsand all is good. I am backing up the Users-Folder as >>>
"$RsyncShareName": "/Users/username" and excluded some unneeded >>>
directories. >>> >>> See macos.pl: $Conf{BackupPCNightlyPeriod} = 1;
>>> $Conf{RsyncShareName} = [ '/Users/username' ]; >>>
$Conf{RsyncSshArgs} = [ '-e', '$sshPath -l username' ]; >>> >> In
reviewing again your settings I see that you have the nightly >>
period configured as a per host value to do a full traverse as I >>
do, but for my system this is configured on the server and not by >>
client. I also see that your: '$sshPath -l username' lists >>
username. For my setup that username is backuppc for all clients. >>
Do you have different usernames configured for each client? >> >> One
thing I've noticed in the nightly process is that even with the >>
setting to traverse the whole pool in one night it seems that I'll >>
get a big prune (4-5) G's late in the weekly cycle. >> >> -- Jim KR >
The other thing that I noticed is that your rsync args is missing the
option:
--protect-args
This may be important to BackupPC's operation. It is recommended that
you not change the rsync arguments from their default in the supplied
config.pl
I was guilty of modifying the defaults and it took me three months to
clear the backups of deleted data across four backup disks.
--
Jim KR
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
On 11/26/23 05:30, Christian Völker wrote:
Hi, > > thanks for pointins this out. I do not remember having
set this > manually on all hosts. It was set globally
anyways so I removed these > entries from all clients. > >
And I have only a username for this single host (as it is
MacOS and > does not know "root" as the others do). > >
Meanwhile I have removed the host an created it from
scratch. Started > a backupc and now everything seems to
work fine. I do not know how > this has happened. > >
Thanks! > > /KNEBB > > > Am 24.11.23 um 17:05 schrieb jbk:
>> On 11/14/23 14:09, Christian Völker wrote: >>> Hi, >>>
>>> I have BackupPC 4.x running on a Debian system. Works
like a >>> champ since years now. >>> >>> One of my clients
is an Apple Mac with current up-to-date MacOS. >>> rsync is
installed: >>> >>> username@macos Documents % rsync
--version rsync version 2.6.9 >>> protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, >>> Wayne
Davison, and others. So far it works fine. It does the >>>
backupsand all is good. I am backing up the Users-Folder as
>>> "$RsyncShareName": "/Users/username" and excluded some
unneeded >>> directories. >>> >>> See macos.pl:
$Conf{BackupPCNightlyPeriod} = 1; >>> $Conf{RsyncShareName}
= [ '/Users/username' ]; >>> $Conf{RsyncSshArgs} = [ '-e',
'$sshPath -l username' ]; >>> >> In reviewing again your
settings I see that you have the nightly >> period
configured as a per host value to do a full traverse as I >>
do, but for my system this is configured on the server and
not by >> client. I also see that your: '$sshPath -l
username' lists >> username. For my setup that username is
backuppc for all clients. >> Do you have different usernames
configured for each client? >> >> One thing I've noticed in
the nightly process is that even with the >> setting to
traverse the whole pool in one night it seems that I'll >>
get a big prune (4-5) G's late in the weekly cycle. >> >> --
Jim KR > The other thing that I noticed is that your rsync
args is missing the option:
--protect-args
This may be important to BackupPC's operation. It is
recommended that you not change the rsync arguments from
their default in the supplied config.pl
I was guilty of modifying the defaults and it took me three
months to clear the backups of deleted data across four
backup disks.
--
Jim KR
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
Hi,
thanks for pointins this out. I do not remember having set this manually
on all hosts. It was set globally anyways so I removed these entries
from all clients.
And I have only a username for this single host (as it is MacOS and does
not know "root" as the others do).
Meanwhile I have removed the host an created it from scratch. Started a
backupc and now everything seems to work fine.
I do not know how this has happened.
Thanks!
/KNEBB
Am 24.11.23 um 17:05 schrieb jbk:
On 11/14/23 14:09, Christian Völker wrote:
Hi,
I have BackupPC 4.x running on a Debian system. Works like a champ
since years now.
One of my clients is an Apple Mac with current up-to-date MacOS.
rsync is installed:
username@macos Documents % rsync --version
rsync version 2.6.9 protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
So far it works fine. It does the backupsand all is good. I am
backing up the Users-Folder as "$RsyncShareName": "/Users/username"
and excluded some unneeded directories.
See macos.pl:
$Conf{BackupPCNightlyPeriod} = 1;
$Conf{RsyncShareName} = [
'/Users/username'
];
$Conf{RsyncSshArgs} = [
'-e',
'$sshPath -l username'
];
In reviewing again your settings I see that you have the nightly
period configured as a per host value to do a full traverse as I do,
but for my system this is configured on the server and not by client.
I also see that your:
'$sshPath -l username'
lists username. For my setup that username is backuppc for all
clients. Do you have different usernames configured for each client?
One thing I've noticed in the nightly process is that even with the
setting to traverse the whole pool in one night it seems that I'll get
a big prune (4-5) G's late in the weekly cycle.
--
Jim KR
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
On 11/14/23 14:09, Christian Völker wrote:
Hi,
I have BackupPC 4.x running on a Debian system. Works like
a champ since years now.
One of my clients is an Apple Mac with current up-to-date
MacOS. rsync is installed:
username@macos Documents % rsync --version
rsync version 2.6.9 protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
So far it works fine. It does the backupsand all is good.
I am backing up the Users-Folder as "$RsyncShareName":
"/Users/username" and excluded some unneeded directories.
See macos.pl:
$Conf{BackupPCNightlyPeriod} = 1;
$Conf{RsyncShareName} = [
'/Users/username'
];
$Conf{RsyncSshArgs} = [
'-e',
'$sshPath -l username'
];
In reviewing again your settings I see that you have the
nightly period configured as a per host value to do a full
traverse as I do, but for my system this is configured on
the server and not by client. I also see that your:
'$sshPath -l username'
lists username. For my setup that username is backuppc for
all clients. Do you have different usernames configured for
each client?
One thing I've noticed in the nightly process is that even
with the setting to traverse the whole pool in one night it
seems that I'll get a big prune (4-5) G's late in the weekly
cycle.
--
Jim KR
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
On 11/15/23 05:26, Christian Völker wrote: Hi, yes, the latest full backup shows files which have been deleted months ago. At the time of backing up the client they did not exist on the client! For testing I did a new "full" backup. It has the same old files in it :\ So I created a new host (by IP instead of name), copied the "macosclient.pl" to "$IP.pl", reloaded backuppc and staarted a new backup on the IP-based host. This backup now only has the files which really do exists on the client. Looks like there is something broken in the inventory of the backups of this host. Any better idea instead of simply removeing all backups from this host and start a new series? /KNEBB Am 14.11.23 um 20:34 schrieb Norman J. Goldstein: Are you saying that your full backup, 666, contains files that *were* not in your local directory at the time of the backup? Here is my speculation. Your BackupPC_nightly is configured to traverse only a portion of the pool each night for all of the machines backed up and is not proceeding far enough to identify the 0 link count for the now deleted file. For full backups BackupPC does a comparison of files already in the pool to those physically on the client and if a file already exists in the pool, instead of downloading the entire file again it increments the file count associated with this machine and moves on. For files that nolonger exist on the client machine it decrements the file count to zero but does not immediately delete it from the client pool count. The next time BackupPC_nightly runs, assuming it traverses the whole pool, it removes the files that have a zero count associated with that client. In the documentation this is covered in the section headed "BackupPC Design". I had to read this section several times to understand how BackupPC_nightly works and also the config.pl related to nightly. My pool is I think very small so I have the nightly configured to traverse the entire pool each time it runs at the beginning of the backup schedule each day. If I've got this wrong then I think those who do know will now clarify my miss conception. -- Jim KR ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
Beyond the safety precaution you mentioned, rsync doesn't delete files at all unless your RsyncArgs include "--delete". There have been a few previous people mailing the list who didn't have --delete in there, so I wonder whether that might be the problem here. On 11/15/23 12:42, Guillermo Rozas wrote: For testing I did a new "full" backup. It has the same old files in it :\ If you check the full log of that backup, does it shows any error or anything suspicious? I had that kind of things happen sporadically when there is a reading error on the client side, and rsync disables deletions for safety. "Reading error" in my case was usually just a symlink for which the linked file was missing. Best regards, Guillermo ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project:https://backuppc.github.io/backuppc/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
> > For testing I did a new "full" backup. > It has the same old files in it :\ > If you check the full log of that backup, does it shows any error or anything suspicious? I had that kind of things happen sporadically when there is a reading error on the client side, and rsync disables deletions for safety. "Reading error" in my case was usually just a symlink for which the linked file was missing. Best regards, Guillermo > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
Hi,
yes, the latest full backup shows files which have been deleted months
ago. At the time of backing up the client they did not exist on the client!
For testing I did a new "full" backup.
It has the same old files in it :\
So I created a new host (by IP instead of name), copied the
"macosclient.pl" to "$IP.pl", reloaded backuppc and staarted a new
backup on the IP-based host. This backup now only has the files which
really do exists on the client.
Looks like there is something broken in the inventory of the backups of
this host.
Any better idea instead of simply removeing all backups from this host
and start a new series?
/KNEBB
Am 14.11.23 um 20:34 schrieb Norman J. Goldstein:
Are you saying that your full backup, 666, contains files that *were*
not in your local directory at the time of the backup?
On 2023-11-14 11:09, Christian Völker via BackupPC-users wrote:
Hi,
I have BackupPC 4.x running on a Debian system. Works like a champ
since years now.
One of my clients is an Apple Mac with current up-to-date MacOS.
rsync is installed:
username@macos Documents % rsync --version
rsync version 2.6.9 protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
So far it works fine. It does the backupsand all is good. I am
backing up the Users-Folder as "$RsyncShareName": "/Users/username"
and excluded some unneeded directories.
See macos.pl:
$Conf{BackupPCNightlyPeriod} = 1;
$Conf{RsyncShareName} = [
'/Users/username'
];
$Conf{RsyncSshArgs} = [
'-e',
'$sshPath -l username'
];
$Conf{BackupFilesExclude} = {
'/Users/username/.Trash' => [
''
],
'/Users/username/Library/Accounts' => [
''
],
'/Users/username/Library/Application Support' => [
''
],
'/Users/username/Library/Assistant' => [
''
],
'/Users/username/Library/Autosave Information' => [
''
],
'/Users/username/Library/Biome' => [
''
],
'/Users/username/Library/Caches' => [
''
],
'/Users/username/Library/Containers' => [
''
],
'/Users/username/Library/Cookies' => [
''
],
'/Users/username/Library/CoreFollowUps' => [
''
],
'/Users/username/Library/Group Containers' => [
''
],
'/Users/username/Library/HomeKit' => [
''
]
};
$Conf{RsyncArgs} = [
'--super',
'--recursive',
'--numeric-ids',
'--perms',
'--owner',
'--group',
'-D',
'--times',
'--links',
'--hard-links',
'--delete',
'--delete-excluded',
'--one-file-system',
'--partial',
'--log-format=log: %o %i %B %8U,%8G %9l %f%L',
'--stats',
'-z',
'-v'
];
Now I took a look at BackupPC and was going to the last backup
(through "Browse backups"). I see the last succesful backup from today.
But when I compare the files in the backup with the ones existing on
my MacOS there is a hughe difference!
There are so many additional file in the backup which I already
deleted weeks, if not months ago!
This is the list of the current backups until the last full one:
669
incr yes 1 2023-11-13 23:17 72.8 0.9
668
incr no 1 2023-11-12 20:06 205.0 2.0
667
incr no 1 2023-11-10 23:03 90.0 3.9
666
full yes 0 2023-11-08 23:49 710.0 5.8
Checking my local directory I do not see many of the files (As I have
them deleted months ago!).
Why do I have files in my backupc which do not exist on the backed up
client?
Thanks for ideas!
/KNEBB
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project:https://backuppc.github.io/backuppc/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project:https://backuppc.github.io/backuppc/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Outdated Files in Backup?
Are you saying that your full backup, 666, contains files that *were*
not in your local directory at the time of the backup?
On 2023-11-14 11:09, Christian Völker via BackupPC-users wrote:
Hi,
I have BackupPC 4.x running on a Debian system. Works like a champ
since years now.
One of my clients is an Apple Mac with current up-to-date MacOS. rsync
is installed:
username@macos Documents % rsync --version
rsync version 2.6.9 protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
So far it works fine. It does the backupsand all is good. I am backing
up the Users-Folder as "$RsyncShareName": "/Users/username" and
excluded some unneeded directories.
See macos.pl:
$Conf{BackupPCNightlyPeriod} = 1;
$Conf{RsyncShareName} = [
'/Users/username'
];
$Conf{RsyncSshArgs} = [
'-e',
'$sshPath -l username'
];
$Conf{BackupFilesExclude} = {
'/Users/username/.Trash' => [
''
],
'/Users/username/Library/Accounts' => [
''
],
'/Users/username/Library/Application Support' => [
''
],
'/Users/username/Library/Assistant' => [
''
],
'/Users/username/Library/Autosave Information' => [
''
],
'/Users/username/Library/Biome' => [
''
],
'/Users/username/Library/Caches' => [
''
],
'/Users/username/Library/Containers' => [
''
],
'/Users/username/Library/Cookies' => [
''
],
'/Users/username/Library/CoreFollowUps' => [
''
],
'/Users/username/Library/Group Containers' => [
''
],
'/Users/username/Library/HomeKit' => [
''
]
};
$Conf{RsyncArgs} = [
'--super',
'--recursive',
'--numeric-ids',
'--perms',
'--owner',
'--group',
'-D',
'--times',
'--links',
'--hard-links',
'--delete',
'--delete-excluded',
'--one-file-system',
'--partial',
'--log-format=log: %o %i %B %8U,%8G %9l %f%L',
'--stats',
'-z',
'-v'
];
Now I took a look at BackupPC and was going to the last backup
(through "Browse backups"). I see the last succesful backup from today.
But when I compare the files in the backup with the ones existing on
my MacOS there is a hughe difference!
There are so many additional file in the backup which I already
deleted weeks, if not months ago!
This is the list of the current backups until the last full one:
669
incr yes 1 2023-11-13 23:17 72.8 0.9
668
incr no 1 2023-11-12 20:06 205.0 2.0
667
incr no 1 2023-11-10 23:03 90.0 3.9
666
full yes 0 2023-11-08 23:49 710.0 5.8
Checking my local directory I do not see many of the files (As I have
them deleted months ago!).
Why do I have files in my backupc which do not exist on the backed up
client?
Thanks for ideas!
/KNEBB
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project:https://backuppc.github.io/backuppc/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
