I think I've discovered a new level of failure. It started off with these 
errors when attempting to rsync larger files:

rsync_bpc: failed to open 
"/home/localuser/mysql/hostname-srv-sql.our_database.sql", continuing: No space 
left on device (28)
rsync_bpc: mkstemp 
"/home/localuser/mysql/.hostname-srv-sql.our_database.sql.000000" failed: No 
space left on device (28)

Now all backups are failing and I see these in the Xfer Error logs:

BackupPC_refCountUpdate: can't write new pool count file 
BackupPC_refCountUpdate: given errors, redoing host hostname #22 with fsck 
(reset errorCnt to 0)
bpc_poolRefFileWrite: can't open/create pool delta file name 
/var/lib/BackupPC//pc/hostname/22/refCnt/tpoolCntDelta_1_-1_0_70674 (errno 28)
bpc_poolRefRequestFsck: can't open/create fsck request file 
/var/lib/BackupPC//pc/hostname/22/refCnt/needFsck70674 (errno 28)
Can't write new host pool count file 

My guess is that the /var/lib/BackupPC/pc partition is the problem. I took 
advice from some rando on the interwebs [1] to put the pc folder on an ssd but 
perhaps it needs more headroom than suggested:

"... Split the storage up on SSD for the pc folder and something more cost 
efficient for cpool such as SMR drives. ...The pc folder in version 4 
essentially only contains the directory structures and references of files that 
they should contain, so it stays very small. However, it is often read from and 
speeds things up remarkably when it is served fast. Much more so than speeding 
up the cpool."

The "pc" folder lives in "/var/lib/BackupPC/pc" on a local ssd with 5GB 
overhead available. The storage pools live on a platter w/70+GB free. Am I 
short-changing the "pc" folder? Does it need to grow significantly during 
backup runs? If so, how much space is suggested?

Unfortunately my monitoring software (NewRelic Infrastructure) does not provide 
much granularity and has previously hidden similar spikes in usage so I don't 
trust its reports, which does not show any capacity violations.

Thank you!

[1] - https://molnix.com/backuppc-version-4-development-allows-better-scaling/
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
BackupPC-users mailing list
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Reply via email to