Re: [BackupPC-users] appending to the Excludes hash
Bowie Bailey wrote:
> On 5/4/2017 10:35 AM, Paul Fox wrote:
> >
> > is there a nice perl way to do something like this? syntax
> > intentionally left vague:
> >
> > $Conf{BackupFilesExclude} += { ... '/home' }
> >
> > i'd like to be able to append to either the '*' catchall array
> > or a share-specific array.
>
> Off the top of my head, you could do it like this:
>
> push @{$Conf{BackupFilesExclude}{'*'}}, '/dir1', '/dir2', '/dir3';
> push @{$Conf{BackupFilesExclude}{'/'}}, '/dir4', '/dir5', '/dir6';
thanks! syntax was perfect.
paul
=--
paul fox, p...@foxharp.boston.ma.us (arlington, ma, where it's 54.9 degrees)
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Nightly does not find some pool files, but they exist
Hi Holger, The directories for the pools do not use the last bit (128 instead of 256 values). It is documented somewhere. So there is top level 10 and 12, but no 11. Same with next level. D9 and D8 go into D8 since there is no D9 directory. > I believe a file '11d9...' should be in 11/d9/, not in 10/d8/ > (off-by-one in > both path components). I would be interested in knowing why this design decision has been made. (in case the devs reads this - not to complain, but to learn) Leaves me back on my original issue. For me it smells like a bug. Unfortunately perl is none of my languages. If I am the only one seeing this, it could be my hard drive as well, the migration caused quite some stress to it. A fresh one is already waiting... PS: Sorry for the DOT, was a copy error I guess -- Many regards, Dieter Fauth :-) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] appending to the Excludes hash
On 5/4/2017 10:35 AM, Paul Fox wrote:
> i guess this question is really about perl syntax, but it might
> be something commonly done by backuppc users, so here goes:
>
> i have the following in my global config:
>
> $Conf{BackupFilesExclude} = {
>'/' => [
> '/proc',
> '/sys',
> ],
>'*' => [
> '.cache',
> '.gvfs',
> 'slocate.db',
> 'ID',
> '*._nobackup_',
> '*.o'
> ]
> };
>
>
> sometimes i wish to augment this list for a particular host:
>
> $Conf{BackupFilesExclude} = {
>'/' => [
> '/home', # /home is backed up separately on this host
> '/proc',
> '/sys',
> ],
>'*' => [
> '.cache',
> '.gvfs',
> 'slocate.db',
> 'ID',
> '*._nobackup_',
> '*.o'
> ]
> };
>
> clearly it would be cleaner not to have to duplicate the entire
> data structure in the host's config file.
>
> is there a nice perl way to do something like this? syntax
> intentionally left vague:
>
> $Conf{BackupFilesExclude} += { ... '/home' }
>
> i'd like to be able to append to either the '*' catchall array
> or a share-specific array.
Off the top of my head, you could do it like this:
push @{$Conf{BackupFilesExclude}{'*'}}, '/dir1', '/dir2', '/dir3';
push @{$Conf{BackupFilesExclude}{'/'}}, '/dir4', '/dir5', '/dir6';
You can also use an array instead of listing the new entries:
push @{$Conf{BackupFilesExclude}{'*'}}, @newstuff;
Untested, so the syntax may not be perfect...
--
Bowie
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] How does BackupPC 4.1.1 calculate Pools size?
Craig-
I set "$Conf{PoolSizeNightlyUpdatePeriod} = 1;" yesterday, and really
didn't see a change in the size discrepancy between what BackupPC reports
and the file system still remains, after letting it run with the updated
config for two nights. The estimate from my last message grew from 26GiB
to 29GiB, while the actual space used on the disk grew only 1GB. Is is
possible that the process is still using the default value of 16 anyway?
The WebUI shows: "Pool is 29.08GiB comprising 779885 files and 16512
directories (as of 5/4 01:00),"
And a check of the base file system shows:
# du -bhs *
52G cpool
2.2G pc
4.0K pool
My log output from last night suggests that the Nightly job really didn't
take very long, and I question if it had time to scan the pool for actual
size used:
2017-05-04 01:00:00 24hr disk usage: 5% max, 5% recent, 0 skipped hosts
2017-05-04 01:00:00 Aging LOG files, LOG -> LOG.0 -> LOG.1 -> ... -> LOG.13
...
2017-05-04 01:00:00 Running 2 BackupPC_nightly jobs from 0..15 (out of
0..15)
2017-05-04 01:00:00 Running BackupPC_nightly -m -P 6 0 127 (pid=21588)
2017-05-04 01:00:00 Running BackupPC_nightly -P 6 128 255 (pid=21589)
2017-05-04 01:00:00 Next wakeup is 2017-05-04 02:00:00
2017-05-04 01:00:01 BackupPC_nightly now running BackupPC_refCountUpdate -m
-s -c -P 6 -r 0-127
2017-05-04 01:00:01 BackupPC_nightly now running BackupPC_refCountUpdate -m
-s -c -P 6 -r 128-255
2017-05-04 01:00:01 admin1 : __bpc_pidStart__ 21604
2017-05-04 01:00:01 admin : __bpc_pidStart__ 21603
2017-05-04 01:00:36 admin : __bpc_pidEnd__ 21603
2017-05-04 01:00:36 BackupPC_nightly now running BackupPC_sendEmail
2017-05-04 01:00:37 admin1 : __bpc_pidEnd__ 21604
2017-05-04 01:00:37 Finished admin1 (BackupPC_nightly -P 6 128 255)
2017-05-04 01:00:40 Finished admin (BackupPC_nightly -m -P 6 0 127)
2017-05-04 01:00:40 Pool nightly clean removed 0 files of size 0.00GB
2017-05-04 01:00:40 Pool is 0.00GB, 0 files (0 repeated, 0 max chain, 0 max
links), 0 directories
2017-05-04 01:00:40 Cpool nightly clean removed 0 files of size 0.00GB
2017-05-04 01:00:40 Cpool is 0.00GB, 0 files (0 repeated, 0 max chain, 0
max links), 0 directories
2017-05-04 01:00:40 Pool4 nightly clean removed 0 files of size 0.00GB
2017-05-04 01:00:40 Pool4 is 0.00GB, 0 files (0 repeated, 0 max chain, 0
max links), 0 directories
2017-05-04 01:00:40 Cpool4 nightly clean removed 603 files of size 0.06GB
2017-05-04 01:00:40 Cpool4 is 29.77GB, 779885 files (0 repeated, 0 max
chain, 4334 max links), 16512 directories
2017-05-04 01:00:40 Running BackupPC_rrdUpdate (pid=21628)
2017-05-04 01:00:43 admin-1 : 2017-05-04 01:00:43 RRD updated: date
1493942400; cpoolKb 0.00; total 719841210.985352; poolKb 0.00;
pool4Kb 0.00; cpool4Kb 30488312.00
2017-05-04 01:00:45 Finished admin-1 (BackupPC_rrdUpdate)
Thanks again for taking a look at this.
On Mon, May 1, 2017 at 10:50 PM, Craig Barratt wrote:
> The nightly pool check (BackupPC_nightly) only traverses a portion of the
> pool each night. See $Conf{PoolSizeNightlyUpdatePeriod}. The default is
> 16, meaning it takes 16 nightly runs to get through the whole pool.
>
> It looks like your installation is quite small, so you could
> set $Conf{PoolSizeNightlyUpdatePeriod} to 1.
>
> Craig
>
> On Mon, May 1, 2017 at 3:48 PM, Ray Frush
> wrote:
>
>> My instance of Backuppc 4.1.1 reports:
>>
>> "Pool is 26.56GiB comprising 764047 files and 16512 directories (as of
>> 5/1 01:00)"
>>
>> However, when I check the file system, I get 'slightly' different numbers:
>>
>> # du -bhs *
>> 50G cpool
>> 2.1G pc
>> 4.0K pool
>>
>>
>> 26GiB vs ~52GB.
>>
>> The target file system is an NFS file system. Is there something I
>> should be doing different to get a more accurate report of the pool size?
>> How does BackupPC calculate the pool size? (I'm trying to grok the source
>> code, but haven't found the method yet.)
>>
>>
>> Thanks
>>
>>
>> --
>> Ray Frush
>> Colorado State University
>>
>>
--
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
[BackupPC-users] appending to the Excludes hash
i guess this question is really about perl syntax, but it might
be something commonly done by backuppc users, so here goes:
i have the following in my global config:
$Conf{BackupFilesExclude} = {
'/' => [
'/proc',
'/sys',
],
'*' => [
'.cache',
'.gvfs',
'slocate.db',
'ID',
'*._nobackup_',
'*.o'
]
};
sometimes i wish to augment this list for a particular host:
$Conf{BackupFilesExclude} = {
'/' => [
'/home', # /home is backed up separately on this host
'/proc',
'/sys',
],
'*' => [
'.cache',
'.gvfs',
'slocate.db',
'ID',
'*._nobackup_',
'*.o'
]
};
clearly it would be cleaner not to have to duplicate the entire
data structure in the host's config file.
is there a nice perl way to do something like this? syntax
intentionally left vague:
$Conf{BackupFilesExclude} += { ... '/home' }
i'd like to be able to append to either the '*' catchall array
or a share-specific array.
paul
p.s. it's a testament to backuppc's stability that i've had my
subscription to this list disabled for almost 10 years!
=--
paul fox, p...@foxharp.boston.ma.us (arlington, ma, where it's 58.5 degrees)
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Nightly does not find some pool files, but they exist
Hi, Dieter Fauth wrote on 2017-05-04 10:20:10 +0200 [[BackupPC-users] Nightly does not find some pool files, but they exist]: > in the log I see many lines like these: > [...] > 2017-05-04 09:04:01 admin : BackupPC_refCountUpdate: missing pool file > 11d9a30eb21b173a1e073e7a86b83f2e count 16 > [...] > The files itself exist though. This is what ls shows: > ls /var/lib/backuppc/cpool/10/d8/11d9a30* > 10/d8/11d9a30eb21b173a1e073e7a86b83f2e I believe a file '11d9...' should be in 11/d9/, not in 10/d8/ (off-by-one in both path components). > Version: 4.1.1git > Repo has been migrated to V4 (I hope all now, just checking right now). A brief look at the code doesn't suggest the migration might be at fault. I wonder how the pool files ended up in the wrong place, particularly since it only seems to apply to some of the files ... Is it ok for migration and refCountUpdate to run in parallel? Regards, Holger P.S.: While "lists.sourceforge.net." (with trailing dot) is a perfectly valid DNS name, at least my MTA doesn't like e-mail addresses ending on a dot. Could you please use the canonical form we all use? -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Display inconsistent for total/existing/new files.
On 5/4/17 12:35 PM, Dieter Fauth wrote: > Hi, > looks like the "existent files" column is way to low in values. > > Example: > 2002 was an V3 backup, 2019 is a V4 backup: > Backup# Type#Files Size/MiBMiB/sec #Files Size/MiB > #Files Size/MiB > 2002 full1736013 94516.1 8.721736008 94491.9 177826.1 > 2019 full2110267 111675.939.67 22 334.6 154 100.9 > > I am the only one seeing this? > I am seeing this as well: https://github.com/backuppc/backuppc/issues/103 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Display inconsistent for total/existing/new files.
Hi, looks like the "existent files" column is way to low in values. Example: 2002 was an V3 backup, 2019 is a V4 backup: Backup# Type#Files Size/MiBMiB/sec #Files Size/MiB#Files Size/MiB 2002full1736013 94516.1 8.721736008 94491.9 177826.1 2019full2110267 111675.939.67 22 334.6 154 100.9 I am the only one seeing this? -- Many regards, Dieter Fauth :-) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Nightly does not find some pool files, but they exist
Hi, in the log I see many lines like these: 2017-05-04 09:03:55 BackupPC_nightly now running BackupPC_refCountUpdate -m -s -c -P 5 -r 16-31 2017-05-04 09:03:55 admin : __bpc_pidStart__ 725 2017-05-04 09:04:01 admin : BackupPC_refCountUpdate: missing pool file 11d9a30eb21b173a1e073e7a86b83f2e count 16 2017-05-04 09:04:01 admin : BackupPC_refCountUpdate: missing pool file 109ea560445eaeaff3b90f2d0810ca95 count 6 2017-05-04 09:04:01 admin : BackupPC_refCountUpdate: missing pool file 1196c64c5ca46448928d9c57189349e5 count 16 ... The files itself exist though. This is what ls shows: ls /var/lib/backuppc/cpool/10/d8/11d9a30* 10/d8/11d9a30eb21b173a1e073e7a86b83f2e Version: 4.1.1git Repo has been migrated to V4 (I hope all now, just checking right now). -- Many regards, Dieter Fauth :-) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
