Hi all,
sorry in advance for a rather long mail which will probably not be of interest
to everyone. I considered replying to backuppc-users, but that doesn't seem
to make much sense either.
Matthias Meyer wrote on 2011-03-12 13:27:24 +0100 [[BackupPC-devel] [Patch]
3.1.
0 partial and yearly backups]:
> [...]
> $Conf{useEveryPartial} and $Conf{PartialAgeMax} will allow to store each
^
(all of the configuration variables without exception start with a capital
letter; let's keep it that way)
> interrupted full backup as well as each interrupted incremental backup as a
> partial backup.
> This partial backups will be used by rsync like incremental backups.
> I see in my environment a dramatical performance improvement at least for
> the initial backup of a new client.
I see why this would be desirable for the initial backup, but ...
1.) Doesn't BackupPC already do this for the initial backup? The initial
backup will always be a full, and failing full backups are stored as
partials and reused as reference - albeit not indefinitely -,
aren't they?
If there is a bug in this implementation (is there?), shouldn't it be
fixed rather than worked around?
2.) Wouldn't your patch introduce a potential dramatic slowdown for later
backups?
Imagine your data set has 1000000 files and 300 GB of data. An arbitrary
backup fails on file number 3 after transferring 1 MB of data because of
a client reboot, network failure, whatever. Using the partial backup as
reference would mean re-transferring almost all of the data, rather than
just discarding the 1 MB that were transfered and doing a normal rsync
delta, would it not?
To fix this correctly, you would need to merge the partial with its
reference backup and use that as a new reference, presuming that is at
all possible (which I doubt).
3.) You need to be aware that the partial backups do *not* represent a
snapshot of your system state that existed at any point in time. They
are broken by definition and only useful - in my opinion - internally
to BackupPC for speeding up the next backup. Once that has completed,
they are obsolete. Why would you want to keep them?
4.) Failed backups are retried at the next wakeup after failing, aren't they?
Wouldn't that mean that you *could* end up storing many failed partials
(e.g. 24 per day) for one host, for which you have scheduled much larger
backup intervals (e.g. daily)?
My knowledge of BackupPC is getting a little rusty, so I might be mistaken on
any of these points, but I'm sure Jeffrey will correct me if so ;-).
> $Conf{FullCntYearly} will allow to make the first backup of a year as an
> full backup and save this full backups an amount of years.
That could be useful for some people, but why limit it to "first backup in a
year"? You might just as well need the first backup of a month, a quarter of a
year, or the first sunday in a year to be a full backup (and retained
specially). You can probably get all of that from a cron job triggering a full
backup and doing some magic to prevent its deletion. Along the lines of
Jeffrey's suggestion, introducing a new backup attribute "no-autodelete" and
allowing to pass that along with the "manual" backup request from cron seems
like a far more general solution.
In your implementation, does $Conf{FullCntYearly}=0 prevent the first backup
in a year from being treated specially (compatibility mode, so to say), i.e.
make it a full or incremental as determined by the normal schedule? Hmm, it
apparently does, but it also seems to schedule the backup for all hosts with
$Conf{FullCntYearly} > 0 for the *first* wakeup in the new year, regardless
of how old the previous backup is. That would mean all hosts to which this
applies run concurrent full backups at the beginning of a year (and presumably
keep in sync pending manual intervention)? Fireworks? :-)
> I've extracted the patch with "diff -ruN <new-file> <original-file>" but
Well, as would be expected, that produces the changes for *removing* your
patch (differences from new to old), which is the wrong way around. This error
is so common that patch detects it and offers to apply the reversed patch.
Hint: in the patch, lines you added should be indicated by a "+" at the
beginning, not a "-" ;-).
> I've never applied this patch => Be carefull.
Why not? It's not difficult. Presuming you have the original files in a tree
like you normally would. If you plan to patch a tree of files, you would
probably be best off to do something like ...
% cd /usr/share
% cp -dpr backuppc backuppc.orig
... edit the files you want to change in the directory backuppc ...
% diff -Nru backuppc.orig backuppc
... instead of creating directories named "_original" and copying files
whenever you start changing them (hopefully not forgetting so). It also
greatly simplifies *creating* the patch, and you can easily *verify* it:
% cd backuppc.orig
% patch -p3 --verbose --dry-run --global-reject-file=../oops.rej <
../b4u-partial+yearly.patch
(presuming you want to strip 3 path components -- usr/share/backuppc in this
case; the diff generated by the command above would need a '-p1' instead).
Aside from that, you shouldn't submit patches reflecting the debian package's
installation layout. Get the source tarball and generate a patch relative to
that. It's really just a matter of matching up files and verifying that the
debian package does not introduce any changes your patch depends upon. Either
you do that work or [possibly several] others have to, and it's probably
easier for you to resolve or rule out any conflicts, because it's your code
changes.
Additionally, you avoid the problem of having two starting directories
(/usr/share/doc/backuppc and /usr/share/backuppc).
All of that said, your patch applies cleanly (with reversed chunks, that is)
to the files from the Sourceforge tarball [erm, well, from the .orig.tar.gz
from Debian, but let's hope that is identical ;-] - once the files are moved
into the places where the patch expects them). On Debian installations with
BackupPC 3.1.0 installed, it should apply as is, but I'd really recommend
building a patched .deb and installing that rather than patching the installed
package.
Sorry if that should sound negative. Just trying to point out some things that
seem important to me.
Regards,
Holger
------------------------------------------------------------------------------
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
BackupPC-devel mailing list
BackupPC-devel@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-devel
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
