> > 2019-04-04 01:18:05 Cpool4 is 4118.27GB, 7700145 files (0 repeated, 0 max > chain, 6883996 max links), 17028 directories
These BackupPC4 cpool stats are normal. Since BackupPC4 uses a full-file digest (unlike BackupPC3), pool collisions (different files with the same digest) are exceedingly rare, and are likely to only happen if you use known methods to create such files. Specifically, "0 repeated, 0 max chain" means there are no hash collisions. "6883996 max links" means there is some file that appears in 6883996 different places across the backups. So it is definitely finding repeated files, and a lot of them. So we need to find out why the BackupPC4 instance appears to be using a lot more resources. It's possible the BackupPC4 configuration is backing up more files than you expect (eg, sparse files, other mounted file systems?). Are you sure your exclude / include rules for each client are the same in v3 and v4? Craig On Fri, Apr 5, 2019 at 10:23 AM G.W. Haywood via BackupPC-users < backuppc-users@lists.sourceforge.net> wrote: > Hi there, > > On Fri, 5 Apr 2019, Stefan Schumacher wrote: > > > ... sorry for ... so many questions ... > > Please don't apologize for asking sensible questions. At the very > least it will help the people who try to document this stuff, and > it might even help to find some of the bugs in new software. :) > > > Old: > > 2019-03-19 06:48:27 Cpool is 1456.03GB, 8695243 files (2303 repeated, > > 373 max chain, 31999 max links), 4369 directories > > > > New: > > 2019-04-04 01:18:05 Cpool4 is 4118.27GB, 7700145 files (0 repeated, 0 > > max chain, 6883996 max links), 17028 directories > > > > Despite backing up the same servers as V3 V4 finds no repeated files > > and no chains. This is most likely the reason why the server is running > > full despite being in use less than 3 month and having triple the > > storage capacity of the old server. In V3 about 80% percent of space > > was used in the cpool, with the other 20% being allocated to pc. > > > > My questions are: Is this the cause of my capacity problems? Is this a > > problem of ZFS? How can I fix this? > > I doubt it's a ZFS problem, but I wouldn't bet money on it. > > I ran into something vaguely similar when I first started to use V4 on > systems which had been backed up using V3 for many years. Look in the > archives for my mails to the list around Sept 14-20 2018 in the thread > > BackupPC 4.2.1 apparently in an infinite loop. > > For me, the BackupPC Web interface claimed that it was using something > like three times the total storage available on the partition, which > obviously was nonsense. The issue was never entirely resolved - in > that the reason for the garbage usage figures was never identified - > but at the suggestion of someone on the list I tweaked a configuration > option and things got back to something like normal fairly quickly. > My feeling now is that V4 is still using rather more storage than V3 > (something like 70% more) to do the same job, but I haven't felt the > need to dig into it because I moved everything else off the partition > which holds the pools, and that left plenty of free space. > > I've occasionally tested recovery of a few backed-up files, there's > never been a problem. I'm not in the habit of losing large chunks of > data so I can't comment on operations more exciting than that. > > -- > > 73, > Ged. > > > _______________________________________________ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ >
_______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
