Folks,
Am 20.06.2016 um 22:01 schrieb Larry Irwin (gmail):
> The scripts I use analyze the rsync log after it completes and then
sftp's a summary to the root of the just completed rsync.
> If no summary is found or the summary is that it failed, the folder
rotation for that set is skipped and that folder is re-used on the
subsequent rsync.
> The key here is that the folder rotation script runs separately from
the rsync script(s).
That is what we found to be important some years ago - do most of the
management outside of the scripts that do rsync.
In particular, our scripts prepare a target backup directory with a name
of the form:
slash.20151122.seq.1379.current
- the name of the area being backed-up is obvious, as is the date.
We use Tower of Hanoi management (see the update at
https://en.wikipedia.org/wiki/Backup_rotation_scheme#Tower_of_Hanoi), so
embed a sequence number in the backup directory name, and the suffix
.current to indicate a current backup target.
we can keep on trying the rsync backups until we get success, and the
scripts then remove the .current suffix. This works because the rsync
includes the --delete option, used since the backup target can be a
recycled hard-linked directory: also recommended.
> For each entity I want to rsync, I create a named folder to identify
it and the rsync'd data is held in sub-folders:
> daily.[1-7] and monthly.[1-3]
> When I rsync, I rsync into daily.0 using daily.1 as the link-dest.
> Then the rotation script checks daily.0/rsync.summary - and if it
worked, it removes daily.7 and renames the daily folders.
> On the first of the month, the rotation script removes monthly.3,
renames the other 2 and makes a complete hard-link copy of daily.1 to
monthly.1
> It's been running now for about 4 years and, in my environment, the
10 copies take about 4 times the space of a single copy.
> (we do complete copies of linux servers - starting from /)
> If there's a good spot to post the scripts, I'd be glad to put them up.
I'd recommend Tower of Hanoi, as referenced above, for managing sets of
backups - far simpler than dealing with days and dates, etc. For
example, the above mentions special actions on the first of the month -
what if that is missed because of some failure - more special case
scripting? Tower of Hanoi is self-healing....
Regards
Rob.
Dr Robert C. Bell
HPC National Partnerships | Scientific Computing
CSIRO Information Management and Technology
T +61 3 9545 2368 (Docklands and Clayton)
T +61 3 9669 8102 (Docklands only)
Mob +61 428 108 333
robert.b...@csiro.au<mailto:robert.b...@csiro.au> | www.csiro.au
https://wiki.csiro.au/display/ASC/Scientific+Computing+Homepage
Also: Bureau of Meteorology Scientific Computing Services
Street: CSIRO SC Lvl 11, 700 Collins Street, Docklands Vic 3008, Australia
Postal: CSIRO SC Lvl 11, GPO Box 1289, Melbourne Vic 3001, Australia
Street: CSIRO IMT, Rivett Room RNW 2.03, Clayton Vic 3168, Australia
Postal: CSIRO IMT, Private Bag 10, Clayton South Vic 3169, Australia
Parcels: CSIRO IMT, Gate 3 Normanby Road, Clayton Vic 3168, Australia
PLEASE NOTE
The information contained in this email may be confidential or
privileged. Any unauthorised use or disclosure is prohibited. If
you have received this email in error, please delete it immediately
and notify the sender by return email. Thank you. To the extent
permitted by law, CSIRO does not represent, warrant and/or
guarantee that the integrity of this communication has been
maintained or that the communication is free of errors, virus,
interception or interference.
Please consider the environment before printing this email.
On 22/06/2016 10:00 pm, rsync-requ...@lists.samba.org wrote:
Send rsync mailing list submissions to
rsync@lists.samba.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.samba.org/mailman/listinfo/rsync
or, via email, send a message with subject or body 'help' to
rsync-requ...@lists.samba.org
You can reach the person managing the list at
rsync-ow...@lists.samba.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of rsync digest..."
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
---------------------------------------
Today's Topics:
1. Re: rsync script for snapshot backups (Dennis Steinkamp)
2. Re: rsync script for snapshot backups (Petros Angelatos)
_______________________________________
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync mailing list
rsync@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
