Hi there,
On Sat, 15 Sep 2018, Michael Stowe wrote:
... 4.2.1 seemed to be eating a LOT more disc space than V3.x ...
tried used BackupPC_backupDelete to delete some cruft ...
And I used "rm -r *" to list the contents of a directory.
Thanks very much for your reply, and if you ever find yourself
confronted by a mob of respectable physicists, run like hell.
... BackupPC_backupDelete is not a tool which removes directories
you did not mean to back up. It deletes entire backups or shares.
Not wishing to contradict you, I'll let the documentation do that:
http://backuppc.sourceforge.net/BackupPC-4.2.1.html#Other-Command-Line-Utilities
states (the emphasis is mine):
"BackupPC_backupDelete deletes an entire backup, OR A DIRECTORY PATH WITHIN A
BACKUP."
Given what I've done, is any of this to be expected?
Yes, it is absolutely to be expected, as the tool does exactly what
it says it does. It does not do what you want it to do ...
Maybe the name of the tool does say something, but the documentation
says it does exactly what I want it to do, and the debug output (given
below) shows it attempting to do it. The tool, however, does not seem
to do what the documentation says it does (the debug output shows it
failing, perhaps because it's looking for non-existent directories).
I suspect that this is a fault in BackupPC_backupDelete, but if it's
just a case of incorrect documentation then it's unfortunate that the
tool apparently breaks, and then also breaks BackupPC -- and that I
had to find out this way.
... which is better accomplished in a different way.
Is this different way documented? If so a pointer would be welcome.
... actual disc usage as reported by the OS (df) isn't changing,
but BackupPC appears to think that it has now used 5TB of a 3TB
disc and the claimed usage is growing. ...
...
Pool is 663.71+4946.35GiB ...
Pool file system was recently at 80% ...
Today's pool figures: Pool is 663.71+6922.70GiB
So the pool is using ~7.5TB?!
The pool is in a partition on a 3TB drive. Apparently it is expected
that if I use BackupPC_backupDelete in the way that's documented (and
in which you say that it is not intended to be used), then not only
will the tool not do what it's documented to do, but also BackupPC may
not recover from the shock -- symptoms of which include what appears
to be an infinite loop which I've yet to break (the incremental backup
which was started on 8th September completed in just under 8 minutes,
but the one started on 9th September is still running a week later);
fanciful calculations of the storage in use by the pools displayed on
the Web interface; and what looks like a propensity to incrementally
delete all the V4 backups that were made in previous runs.
tornado:/var/lib/backuppc/pc# >>> date
Sun 16 Sep 13:04:38 BST 2018
tornado:/var/lib/backuppc/pc# >>> tail localhost/LOG.092018
2018-09-07 20:17:25 Finished BackupPC_backupDelete, status = 0 (running time:
104 sec)
2018-09-08 20:00:02 incr backup started for directory Config
2018-09-08 20:00:27 incr backup started for directory Homes
2018-09-08 20:08:23 incr backup 735 complete, 916597 files, 190430491468 bytes,
0 xferErrs (0 bad files, 0 bad shares, 0 other)
2018-09-08 20:08:23 Removing unfilled backup 720
2018-09-08 20:08:23 BackupPC_backupDelete: removing pre-v4 backup #720
2018-09-08 20:10:07 BackupPC_backupDelete: got 0 errors
2018-09-08 20:10:07 Finished BackupPC_backupDelete, status = 0 (running time:
104 sec)
2018-09-09 20:00:04 incr backup started for directory Config
2018-09-09 20:00:41 incr backup started for directory Homes
You can, of course, delete the entire backup, then set excludes, and
back up what you want. This is probably simplest if you're going to
fool around at the storage level.
Yes, of course I can, but I'll lose a lot that way. Just to be clear,
my intention was not to "fool around", but to prevent BackupPC V4 from
using MORE THAN THREE TIMES AS MUCH STORAGE as BackupPC V3 used TO DO
EXACTLY THE SAME JOB and in the process run out of space on the backup
partition. The partition was originally sized three times bigger than
needed when it was running BackupPC V3, but is now proving to be much
less than adequate. So much so that I might as well go back to using
rsync from a cron job (and which, incidentally, I still do for all the
really important stuff).
... don't try to screw around with the contents of individual
backups. It's possible, but you'll need to have a deep
understanding of what tools to use and the ramifications ...
Even supposing that my usage of BackupPC_backupDelete was incorrect,
one would hardly expect that to throw the backup service into a fit.
I've done things like this for years with V3. Apparently V4 is less
forgiving than V3 in this respect.
Below is the debug output mentioned earlier in this message.
8<----------------------------------------------------------------------
BackupPC_backupDelete debug output: userid is redacted, no other changes.
The pool is on a 3TB LUKS encrypted volume which is mounted on /mnt/3TLV
and /var/lib/backuppc is a symlink to /mnt/3TLV/backuppc. The function
RmTreeQuietInner seems to be seeing some sort of bowdlerization of these.
8<----------------------------------------------------------------------
tornado:/var/lib/backuppc/pc/kestrel# >>>
/usr/share/backuppc/bin/BackupPC_backupDelete -h kestrel -n 268 -l -s Homes XXXXX/.cache
'XXXXX/.moonchild productions'
__bpc_pidStart__ 13276
BackupPC_backupDelete: $fullDelPath=[/Homes/XXXXX/.cache];
$delPathM=[/XXXXX/f.cache]
BackupPC_backupDelete: removing #268/Homes/XXXXX/.cache
__bpc_progress_state__ merge #268/Homes/XXXXX/.cache -> #267/Homes/XXXXX/.cache
BackupPC_backupDelete: Merge into backup 267/Homes/XXXXX/.cache
bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, KeepOldAttribFiles = 0
RmTreeQuietInner:
/mnt/3TLV/backuppc/pc/kestrel/268//var/lib/backuppc/pc/kestrel/268/XXXXX isn't
a directory (while removing /var/lib/backuppc/pc/kestrel/268/XXXXX/f.cache)
BackupPC_backupDelete: $fullDelPath=[/Homes/XXXXX/.moonchild productions];
$delPathM=[/XXXXX/f.moonchild productions]
BackupPC_backupDelete: removing #268/Homes/XXXXX/.moonchild productions
__bpc_progress_state__ merge #268/Homes/XXXXX/.moonchild productions ->
#267/Homes/XXXXX/.moonchild productions
BackupPC_backupDelete: Merge into backup 267/Homes/XXXXX/.moonchild productions
RmTreeQuietInner:
/mnt/3TLV/backuppc/pc/kestrel/268//var/lib/backuppc/pc/kestrel/268/XXXXX isn't
a directory (while removing /var/lib/backuppc/pc/kestrel/268/XXXXX/f.moonchild
productions)
bpc_attrib_dirWrite: old attrib has same digest; no changes to ref counts
__bpc_pidStart__ 13279
BackupPC_refCountUpdate: computing totals for host kestrel
__bpc_progress_state__ sumUpdate 0/128
bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, KeepOldAttribFiles = 0
__bpc_progress_state__ sumUpdate 8/128
__bpc_progress_state__ sumUpdate 16/128
__bpc_progress_state__ sumUpdate 24/128
__bpc_progress_state__ sumUpdate 32/128
__bpc_progress_state__ sumUpdate 40/128
__bpc_progress_state__ sumUpdate 48/128
__bpc_progress_state__ sumUpdate 56/128
__bpc_progress_state__ sumUpdate 64/128
__bpc_progress_state__ sumUpdate 72/128
__bpc_progress_state__ sumUpdate 80/128
__bpc_progress_state__ sumUpdate 88/128
__bpc_progress_state__ sumUpdate 96/128
__bpc_progress_state__ sumUpdate 104/128
__bpc_progress_state__ sumUpdate 112/128
__bpc_progress_state__ sumUpdate 120/128
__bpc_progress_state__ rename total
BackupPC_refCountUpdate: host kestrel got 0 errors (took 8 secs)
BackupPC_refCountUpdate total errors: 0
__bpc_pidEnd__ 13279
BackupPC_backupDelete: got 0 errors
__bpc_pidEnd__ 13276
8<----------------------------------------------------------------------
Once again, thank you for your input. Do consider some restraint.
--
[Reply delayed until 12:20GMT in case of more on the digest list.]
73,
Ged.
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
