Re: [BackupPC-users] Backup problems with /home directory
Hi once more, I haven't fully followed, but I think that you've written that the sudoers file has the following entry: # setup for backuppc to run rsync backuppc ALL= NOPASSWD:/usr/bin/rsync --- and at some other point you mentioned that rsync is run by user 'nobody'...? _Not_ backuppc...? So this sudo entry might be useless? You'll figure it out, just take it easy ;) Look for what user on which end is doing what, one single step after another. Regards, Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Backup problems with /home directory
Hi there, On the client system I can sudo to the backuppc user and, as a test I ran "sudo rsync /home/john/testfile.txt /home/backuppc/" and it succeeds with no errors. The file was copied to the backuppc home directory. But it always fails when being run from the BackupPC server with the "Permission denied (13)" error. I'm stumped and any help would be appreciated. Any ACL (Access Control List) in place...? SELinux enabled? Maybe the sudo comand run by the server differs from the one that you are testing? What if you vary the permissions on the problematic directories for a single run/day? I'm curious about what you will find. ;) Good luck! Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] How to run a manual backup.
Thanks, yes the real problem is with the configuration of sshd on CS-9. It seems to be slightly different than on CS-8. So I don't have ssh working on the CS-9 box yet. I get the following error when I try to ssh between accounts even on the laptop box itself. sshd[2608]: refusing RSA key: Invalid key length [preauth] sshd[2608]: Accepted password for root from 192.168.1.8 port 35958 ssh2 So I was hoping for the test of BackupPC that running a manual backup when the ssh login failed that it would fall back and ask for a password. This is just a test laptop and I wanted to make sure all the services I regularly use were working on it before I upgraded my home network. So it's back to sorting out sshd. Have a look for that on https://serverfault.com/questions/1095898/how-can-i-use-a-legacy-ssh-rsa-key-on-centos-9-stream Maybe it helps. Iosif Fettich___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] $Conf{RsyncBackupPCPath} is set to , which isn't a valid executable
Hi,
I am sure I have $Conf{RsyncBackupPCPath} set properly
# grep RsyncBackupPCPath *
config.pl:$Conf{RsyncBackupPCPath} = "/usr/libexec/backuppc-rsync/rsync_bpc";
config.pl:RsyncBackupPCPath => 0,
# file /usr/libexec/backuppc-rsync/rsync_bpc
/usr/libexec/backuppc-rsync/rsync_bpc: ELF 64-bit LSB pie executable, x86-64,
version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
BuildID[sha1]=9476c80ebc0ea2113223b0f94b5d9745c43aeafe, for GNU/Linux 3.2.0,
stripped
I'm not sure if this would help or if is irrelevant, but on one of my
servers I have
# grep RsyncBackupPCPath /etc/BackupPC/config.pl
$Conf{RsyncBackupPCPath} = '/usr/bin/rsync_bpc';
'RsyncBackupPCPath' => '0',
# file /usr/bin/rsync_bpc
/usr/bin/rsync_bpc: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
BuildID[sha1]=f87f6bc67ba74346a66641cb2e45be104456e6eb, for GNU/Linux 3.2.0,
stripped
# cat /etc/system-release
Fedora release 38 (Thirty Eight)
# rpm -qa | grep -i backuppc
BackupPC-XS-0.62-11.fc38.x86_64
BackupPC-4.4.0-9.fc38.x86_64
and on another one that is
# grep RsyncBackupPCPath /etc/backuppc/config.pl
$Conf{RsyncBackupPCPath} = '/usr/libexec/backuppc-rsync/rsync_bpc';
'RsyncBackupPCPath' => '0',
# file /usr/libexec/backuppc-rsync/rsync_bpc
/usr/libexec/backuppc-rsync/rsync_bpc: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=9476c80ebc0ea2113223b0f94b5d9745c43aeafe, for GNU/Linux 3.2.0, stripped
# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"
# dpkg -l | grep -i backuppc
ii backuppc 4.4.0-5ubuntu2
amd64high-performance, enterprise-grade system for
backing up PCs
ii backuppc-rsync3.1.3.0-3build1
amd64patched rsync for BackupPC version 4
ii libbackuppc-xs-perl:amd64 0.62-2build1
amd64Perl module with C backend for BackupPC 4
Best regards,
Iosif Fettich
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] File Size Summary shows too large value for host
Hello Stefan, Just to be sure, what do you mean by "Can you check sizes in parallel"? Doing the very same set of backups, at the same time, using different servers. I have already looked at the two backups, from v3 and v4, and compared them. There are NO differences in the exports. Same size, same content, everything identical. Definitely _eveything_ identical? In the logs you used as example previously, there were _different_ files skipped as incomplete showing up. If I start with a smaller list of files for the backup, I still see a difference. However, the difference becomes smaller. What if you start with a single file to be backed up? And/or keep the amount of files sufficiently small to be able to sum up the sizes manually to see if the total fits? Are the two servers running the same OSes? Might be far fetched, but maybe the filesystems are using different block sizes on disk? Best regards, Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] File Size Summary shows too large value for host
Hello Stefan, I understand that v3 and v4 work differently. However, I can also reproduce this behavior when the two hosts have no history of backups at all. Starting with the first full backup, the size difference is shown. I have other hosts where this problem does not occur, however I cannot find the difference. That's interesting. Can you check sizes in parallel? Going from top to bottom, comparing the biggest directories first, entering the one that shows the biggest size diff, etc.? If there is a big difference, I do not really understand how you cannot find at least one particular file or dir co-responsible for that difference. It takes some time, of course, but since this happens only on some of your hosts, I suspect you can solve the mistery only by hunting this down on these affected hosts. The affected hosts do not have particularly large sparse files, they do not have bind mounts, no excessive number of hard links. What if you start back-ups repeatedly from scratch (and at the same time...), but increasing steadily the dirs you do backup? I'd probably start only with /home, excluding /tmp, /mnt/, /var/, ... Would you still see size differences...? Btw: what's your set-up, do you have different servers for BacupPC v3 and Backup PC v4? Thanks, Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] File Size Summary shows too large value for host
Hello Stefan, But how can I explain that both backups are identical in size and content as soon as I export them as .zip or .tar? At most, it happens that the backups differ in one or two lines in a log file, but are otherwise completely identical. Nevertheless, BackupPC v4 displays something completely different in the overview. I can just suppose that is is related to the following: (https://backuppc.sourceforge.net/BackupPC-4.0.0.html): [...] This is the first release of 4.0, which is a significant rewrite of BackupPC. This section provides a short overview of the changes and features in 4.0. Here's a short summary of what has changed in V4: No use of hardlinks (except temporarily to do atomic renames). Reference counting is handled at the application level in a batch manner. Backups are stored as "reverse deltas" - the most recent backup is always filled and older backups are reconstituted by merging all the deltas starting with the nearest future filled backup and working backwards. This is the opposite of V3 where incrementals are stored as "forward deltas" to a prior backup (typically the last full backup or prior lower-level incremental backup, or the last full in the case of rsync). [...] I think you cannot properly compare v3 with v4 other then starting from a system that was never backed up before. .zip and .tar do not know anything about the history of the files they deal with, or about previous backups. In contrast, BackupPC focuses and puts a lot of effort on keeping track of what was previously seen already, to avoid duplication. Good luck, Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] File Size Summary shows too large value for host
Hello Stefan, here I have the logs from two different hosts, each backed up with v3 and v4. Unfortunately, both hosts have been set up in BackupPC for a while, so they are not completely fresh backups. However, I don't think that should make a difference. You can see that despite identical numbers of files, the backups have significant differences in size. Identical number of files != same files v3 2022-12-12 10:15:15 full backup started for directory / (baseline backup #4521) 2022-12-12 10:17:31 full backup 4522 complete, 75368 files, 4031524511 bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other) Remote[2]: file has vanished: "/var/spool/exim4/input/1p4eu4-0002rf-1f-D" [...] Remote[2]: file has vanished: "/mnt/system/tmp/cron-apt.klojST/initlog" Remote[2]: file has vanished: "/tmp/cron-apt.klojST/initlog" Since you're backing up temporary and ephemeral files as well, besides the ones that you could actually use, it's very possible that you get some files that are changing allot in size within no time. Archived logfiles can change from many GBs to just a handful of bytes whenever logrotate steps in... Look after /var/log/messages* for example and compare the sizes you see with the two types of backups. Just a thought: backups are _not_ like snapshots, if you think about catching a particular (and reproducible...) state of your machine. Best regards, Ioisf Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] File Size Summary shows too large value for host
Hello Stefan, you do understand correctly. I'm doing the same backup with v3 and v4. No significant time delay between backups. I would be ok with slight differences, temp files having changed, log files having a few more lines, but i see differences in the gigabyte range. What makes it even more complicated: When exporting the backup as .tar or .zip it is almost the same size (not counting the slight differences of logs, temp files, etc). The significant size difference is only in BackupPC. There was recently some mentions about BackupPC v4 behaving differently by default than v3 with regard to hardlinked files. I haven't followed that in detail - maybe that's what you're seeing? As fas I understood, that would imply that hardlinked files would appear as missing in v4 and would _not_ be included in the backups [by default], whereas they would show up normally in v3. You might still need to look for some specific differences to see if that's what causes the issue you're seeing. Best regards, Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] File Size Summary shows too large value for host
Hello Stefan, i can already reproduce this with a single backup. The following initial situation: A host that has never been backed up. Now BackupPC v3 and v4 both make a full backup, but end up showing very different results as far as the size of the backup is concerned. Does this describe my question more clearly? Sort of :) Sounds like a different problem this time, as you - correctly - do not mention 'du' anymore. If I understand correctly, you see different sizes when using BackupPC v3 versus doing the _very same_ backup using BackupPC v4. Is that it? The files that were backuped do not differ, was the system not running between the two backups? No temporary files - that might differ significantly in size! - created or deleted between the run of v3 and v4? (and not being excepted from the backups) I'm not aware that v4 would bring significant differences in the size of the archives when compared with v3, but I might be wrong. I'd try to look in some detail on the backuped files in order to spot the differences. Best regards, Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] File Size Summary shows too large value for host
Hallo Stefan,
i just wanted to ask if nobody has an idea about this? Where can the size
difference come from?
Greetings
Hello all,
I have a strange problem with my BackupPC 4.4.0 installation.
With two hosts to be backed up, the "File Size/Count Reuse Summary" shows a too large value
for "Totals -> Size/MiB".
The system to be backed up is about 77 GiB ("du -sh /" shows 77G).
Nevertheless, BackupPC shows 106110 MiB.
Where can this difference in size come from? In an old installation, with
BackupPC 3.3.2, the size is displayed correctly.
How can I investigate this error? What information is needed to investigate
further?
Why do you think this is an error...? BackupPC keeps a number of backups,
some full, some incremental - how many exactly you'll have to check in
your config.
Your files will change at some rate.
Assuming you'd keep 7 backups and do completeley rewrite your disk
every day, then in a week your backups will increase to 7 x 77 GB,
although du will still show only 77 GB used...
Is that your problem? Seems to be just a misunderstanding.
Best regards,
Iosif Fettich
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Invalid config.pl via configuration web interface
Hi Alexander,
here's what probably has bitten yoou:
---
Date: Fri, 15 Apr 2022 11:45:54 -0700
From: Craig Barratt
Reply-To: backuppc/backuppc
To: backuppc/backuppc
Cc: Iosif Fettich , Mention
Subject: [backuppc/backuppc] Config write fails with Data::Dumper versions >=
2.182 (Issue #466)
Parts/Attachments:
1 OK 11 lines Text
2 Shown23 lines Text
@ifettich [github.com] discovered that config file writing fails with
Data::Dumper versions > 2.178 due to a typo in the Data::Dumper->new()
call. The second argument is missing a qw() wrapper. This was benign up to
around Data::Dumper version < 2.182, but some changes to the XS library
since then expose the long-time bug in BackupPC.
Because Data::Dumper is used in terse mode, there's no need to provide the
variable name in the second argument. So the fix is to simply remove the
2nd argument. That fix is backward compatible with older versions of
Data::Dumper.
---
Hope this helps a little bit. Mosty probably updating your BackupPC to the
corrected version is all you need to do (besides restoring the settings
that you had in use)
Best regards,
Iosif Fettich
On Wed, 9 Nov 2022, Alexander Kobel wrote:
Dear all,
I receive validation errors of my config file after changes to the (global)
config in the web interface. Consequently, BackupPC terminates.
I'm absolutely sure that this worked before; my last (host) config change dates
back to Feb 2022, the last global config change happened mid 2020.
Unfortunately, I can't pinpoint a specific culprit (system) update anymore.
Normal operation is not affected, so I didn't spot the issue earlier; just
undoing the most recent perl-related updates from today's regular update does
not help.
The issue is that upon changing the main config or, e.g., adding a host, HASH
or ARRAY entries in the config file are written with parentheses rather than
braces or brackets, as expected. In turn, I receive
Software error:
Not an ARRAY reference at /usr/share/backuppc/lib/BackupPC/CGI/Lib.pm line 468.
or similar messages on operations that re-read the config, accompanied by
crashes of the server. Attached is a diff of the config folder, with entries
like
2431c2431
< $Conf{ClientShareName2Path} = {};
---
$Conf{ClientShareName2Path} = ();
2433c2433
< $Conf{RsyncIncrArgsExtra} = [];
---
$Conf{RsyncIncrArgsExtra} = ();
I can fix the config manually and the server starts again; however, I'm not
100% confident whether some log/configuration data is written periodically,
e.g. on nightlies, and more dragons hide behind the scenes.
Did anyone experience a similar problem? Any know incompatibilities with one of
the more recent perl packages? Any clues what might be the problem?
For context, I'm on Arch, pretty much up-to-date; relevant versions of
BackupPC, web server and dependencies are
backuppc 4.4.0-5
lighttpd 1.4.67-1
glibc 2.36-6
popt 1.18-3
perl 5.36.0-1
par2cmdline 0.8.1-2
perl-archive-zip 1.68-7
perl-io-dirent 0.05-15
perl-file-listing 6.15-2
perl-time-modules 2013.0912-8
perl-cgi 4.54-2
perl-xml-rss 1.62-1
perl-json-xs 4.03-3
postfix 3.7.3-2
Thanks and cheers,
Alex
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] BackupPC failed after upgrading client to Debian 11
Hi Taste-Of-IT, and thanks for the hint, but it didnt work. I tried =30 but got the same error. I have actually no clue to solve this. Maybe check if TCPKeepAlive would help. My doc says default is yes, but who knows... Maybe you can run the command manually at some point, while sitting near a monitoring console. I'd probably try that, with some pings or traceroute running in parallel. I think that ServerAliveInterval is for protocol-level keepalives, whereas you might fight a TCP level failure between some intermediate routers. Best regards, Iosif Fettich ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Needed patch for BackupPC::Storage::Text.pm
Hi Ged,
$ grep -n 'my $d = Data::Dumper->new'
/usr/local/BackupPC/lib/BackupPC/Storage/Text.pm
379:my $d = Data::Dumper->new([$newConf->{$var}], [*value]);
413:my $d = Data::Dumper->new([$newConf->{$var}], [*value]);
Ouch. I haven't even noticed that there are two occurencies, the one I
patched was enough for what I was doing.
Both should be corrected in the same way, i.e. no second call argument.
Thanks,
Iosif
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Needed patch for BackupPC::Storage::Text.pm
Hi Ged,
just to let you know:
BackupPC::Storage::Text.pm needs a patch.
in sub ConfigFileMerge, the line reading
my $d = Data::Dumper->new([$newConf->{$var}], [*value]);
should ...
Which one?
I'm not sure what you're asking... I thought giving the full line would be
better than indicating line numbers, as different versions will probably
differ in that regard.
But here's the diff on my system for file
/usr/share/BackupPC/lib/BackupPC/Storage/Text.pm that came with rpm package
BackupPC-4.4.0-5.fc35.x86_64:
$ diff Text.pm Text.pm.BAD
425,427c425
< # my $d = Data::Dumper->new([$newConf->{$var}], [*value]);
< # see https://www.perlmonks.org/?node_id=11142844
< my $d = Data::Dumper->new([$newConf->{$var}]);
---
my $d = Data::Dumper->new([$newConf->{$var}], [*value]);
I've tried to get in touch with Craig Barrat (the BackupPC author and
maintainer, afaik), but haven't succeded yet.
I think you have, now. :)
Are you saying that Craig is following the list? I'm not an active member
and just subscribed, specifically to be able to warn about the issue.
Thanks,
Iosif
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
[BackupPC-users] Needed patch for BackupPC::Storage::Text.pm
Hi BackupPC list members,
just to let you know:
BackupPC::Storage::Text.pm needs a patch.
To be more exact, within sub ConfigFileMerge, the line reading
my $d = Data::Dumper->new([$newConf->{$var}], [*value]);
should become
my $d = Data::Dumper->new([$newConf->{$var}]);
(just omit tle ast argument in the call).
That's a timebomb sitting there, as the existent code works fine only as
long as the installed Data::Dumper on the system is somewhat older,
(probably) having version below 2.182_51.
(See a somewhat more detailed discussion about it at
https://www.perlmonks.org/?node_id=11142844.)
Without the patch, as soon as Data::Dumper will be upgraded/refreshed, any
further attempt to edit BackupPC config variables will ruin any config.pl
files touched within /etc/BackupPC.
Happend to me on an up-to-date Fedora 35 system.
I've tried to get in touch with Craig Barrat (the BackupPC author and
maintainer, afaik), but haven't succeded yet.
Thanks,
Iosif Fettich
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
