On 13/1/20 11:19, Laurence Hurst wrote:
Ahhh, I think I know what‘s going on now. I run Debian stable on all
my systems so still on version 3, yes.
It’s been several years since I copied the files over, before the old
disks were sent for shredding, so they’ve been sitting there as
disabled hosts and I’ve only now got around to finally sorting it out.
Reading back over the documentation and searching the list archive, I
think what I did was to just copied (preserving hardlinks - I am very
familiar with the issues of copying entire, but not partial, BackupPC
stores having done this many times, and currently do so monthly for
offsite/DR purposes) the pc directories, so although the duplicated
files were still hardlinked together (as they came from the same
backuppc filesystem) nothing I copied over will be in this system’s
pool (this process was suggested on this mailing list circa. 2012). I
know I followed someone else’s advice on this for precisely the
reasons you describe and this is the first, and I expect will be the
only time, I’ve tried to partially or selectively access a subset of
data in a BackupPC store.
What I was tying to do at the time was resurrect access to these
specific hosts, so preserving hardlinks within there pc directories
but not adding to the live system pool was sufficient (and within my
understanding of the storage structure ;) ). You’re spot on that
trying to merge the pools myself is beyond my current understanding of
the storage structure, with I don’t “really really understand”.
Assuming this is what I did, that makes sense of what I’m seeing now,
if I’m getting an approx. 50% hit rate on files already inthe pool, on
this live BackupPC, from other hosts and their backups - 50% is higher
than I would have presumed but still plausible. Having foundthe old
mailing list post suggesting just copying the pc directories (with the
caveat that it loses the benefits of pooling for new backups) I’m 90%
sure this is what’s going on, I may try and confirm it tomorrow (if
that’s right, nothing from these old pc directories will exist in the
pool).
I’ll have to modify my plan and work through one old host at a time,
instead of restoring all the data at once then sorting it. I was
expecting (having clearly forgotten how I got these old systems
backups into my current BackupPC install) restoring all the old
backups to my current desktop system would have negligible impact on
the pool’s freespace and I could tidy up the defunct pc’s
configuration and pc directories before I’d made a significant dent in
sorting throughthe restored files. Hopefully doing it a host at a time
then removing that hosts configuration and pc directory will release
enough space from files not shared between the old hosts to keep at
least one backup copy (old or new) of all of the data as I go, just in
case.
Thanks for taking the time to reply. I wouldn’t have found theold
message and realised the above hypothesis, which makes sense of the
situation, without it.
I would suggest the following should work, with minimal issues/disk
space issues.
1) Copy (with hardlinks) the required pc directories from your old
backup server to the new server. Ensure that none of the backup numbers
conflict with existing backup numbers on the new server. Ideally, you
would have no old backups on the new server to begin with.
2) Do the restore from BPC as needed.
3) Delete everything you copied in step 1 above
4) Do a full backup on the server the files were restored to in step 2
You could do step 4 before step 3 but you will need sufficient space on
the new BPC server, which you don't have based on your original post.
Also note that the default BPC config will stop doing backups when the
used disk space is 95% or higher, so be careful/check that scheduled
backups of all your other live servers is still happening as needed.
Regards,
Adam
Laurence
On 12 Jan 2020, at 22:57, backu...@kosowsky.org wrote:
Assuming you are using BackupPC 3.x... which is based on hardlinks...
You can't "just" copy over and merge backups... Rather there are links
and pool/cpool chains and several other complexities.
I did write and post some routines that can do this by essentially
looking up each file, searching the pool for a match, and then either
replacing the 'pc' file with a hard link to an existing pool file or
creating a new link depending on whether the file already exists in
the pool. There are also several edge cases to be careful of.
This is of course a slow process and not recommended unless
you really really understand the structure of the backup storage and
know what you are doing (which clearly you don't :)
Laurence Hurst wrote at about 19:11:02 +0000 on Sunday, January 12, 2020:
Hi I'm hoping someone can help me figure out what I've done and/or where
my expectations are wrong.
Bit of background: I've found some old disks that belonged to a BackupPC
pool, which contained amongst other things copies of some coding
projects I thought I'd lost the source for many years ago. I've copied
them into my existing backuppc pool, along with the relevant pc
directories and configurations, so I can access them.
I've started restoring some of the file to a directory onto my current
machine, using a direct restore. I was expecting to be able to restore
the files, rescue what I needed and delete the original 'pc' from
BackupPC without having much impact on the pool since all of the files
are already in there, so interim backups of my current machine should
just create new links to the files I've recovered via BackupPC's restore
feature. However, when the next backup of my current machine happened
only about 50% of the files were found as 'existing files', the other
50% came out as 'new files' (and now my pool disk is 95% full as nearly
200GB of data that should already be in the pool has been duplicated).
This is where I'm confused: Why would files restored directly from
BackupPC and not subsequently touched in any way (not even viewed or
moved/renamed) not be found as existing files in the pool? Compression
was turned on for both pools (the original and the new one).
Any ideas?
Thanks,
Laurence
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
