I just started to look at / implement your script.
Very impressive. If it is not already common practice among rdiff-backup users to use it, they should give it a look. (I already had ruby installed, so I had no dependency issues. May be a little more trouble for people that have to install ruby.)
FYI: I find it interesting that what I'm backing up is a small intranet Ruby on Rails site and that your script is written in Ruby. We're still learning basic ruby around here, so it will be interesting to go thru your script and try to understand it.
Greg
On 10/19/06, David Kempe <[EMAIL PROTECTED]> wrote:
Hi Greg,
After much experimentation with various scripts, I am happy with this
latest rdiff-backup parser we have written. Script is written by
Christian Marie, who also wrote the rdiff-backup nagios plugin on
nagiosexchange.org.
The script is designed to be passed on stdin the output of an
rdiff-backup job or multiple jobs with -v5 or greater and
--print-statistics both turned on. It can deal with many jobs passed in
ie: backup.sh | parse-rdiff-backup
where backup.sh might run three jobs.
check it out - you get a nicely formatted email that says things like
Backup(s) OK 3/3 in the subject, and a summary of the job like so:
Status of backup(s) follow
Backup completed successfully
Direction: /home/sol1backup/cookie-data to /var/backupdisk/cookieman
Backup started: Tue Oct 17 23:00:06 2006
Source files: 61248
Source files size: 22290479056 (20.8 GB)
New files size: 6224482 (5.94 MB)
Deleted files size: 308229 (301 KB)
Destination size change: 18.4 MB
Errors: 79
Anway, Here is the script and config file. Read the source if you want
more info.... :)
Script:
--------------------------------
#!/usr/bin/env ruby
require 'yaml'
require 'tempfile'
class Parser
attr_reader :status
DEFAULT_CONFIG = File::SEPARATOR +
File.join('etc', 'parse-rdiff-backup.conf')
BACKUPOKCORRECT = /TotalDestinationSizeChange/
def initialize data, config = DEFAULT_CONFIG
begin
@log = data
@config = YAML::load_file(config)['parser']
@statistics_output = []
@statistics = []
@config['statistics'].each do |key,value|
@statistics << SessionStatistic.new(key, value)
end
parse!
rescue Exception => exception
@status = BackupStatus::ERROR
@error = "Error: #{exception.class} (#{exception.message})\n"
@error += "Backtrace:\n"
@error += exception.backtrace.join "\n"
end
end
def parse!
@status = BackupStatus::BROKEN
@log.each do |line|
@ statistics.each do |statistic|
if line =~ /#{statistic.regex}/
if $1
@statistics_output << statistic.message + $1
end
end
end
if line =~ BACKUPOKCORRECT
@status = BackupStatus::OK
end
end
end
def statistics_output
return @statistics_output.join("\n").gsub(/^/, "\t")
end
def error
return @error.gsub(/^/, "\t")
end
private
class SessionStatistic
attr_accessor :message, :regex
def initialize message, regex
@regex = regex
@message = message + ': '
end
end
end
class BackupStatus
OK = 0
BROKEN = 1
ERROR = 2
end
class Message
attr_accessor :subject, :message, :to, :cc, :log
DEFAULT_CONFIG = File::SEPARATOR +
File.join('etc', 'parse-rdiff-backup.conf')
def initialize config=DEFAULT_CONFIG
@config = YAML::load_file(config)['message']
@subject = 'Unknown status'
@message = "Status of backup(s) follow\n\n"
@to = @config['main_contact']
@cc = @config['cc_contact']
end
def send
temp = Tempfile.new "rdiff_backup_log"
temp.puts @log
temp.close
zipped_log = "#{ temp.path}.zip"
`zip #{zipped_log} #{temp.path}`
IO.popen(
'nail ' +
"-a '#{zipped_log}' " +
"-c '[EMAIL PROTECTED]' " +
"-s '[EMAIL PROTECTED]' '[EMAIL PROTECTED]'",
'w'
) do |nail|
nail.puts @message
end
File.unlink zipped_log
end
end
def split_log log
marks = []
logs = []
log.each_with_index do |line, index|
if line =~ /Starting mirror|increment\ operation (.*) to (.*)/
marks << index
end
end
marks.size.times do |index|
first = marks[index]
last = marks[index+1] || log.size
logs << log[first..last].to_s
end
return logs
end
if __FILE__ == $0
parsers = []
input = ARGF.readlines
split_log(input).each do |log|
parsers << Parser.new(log)
end
ok_backups = 0
message = Message.new
parsers.each do |parser|
case parser.status
when BackupStatus::OK
ok_backups += 1
message.message += "Backup completed successfully\n"
message.message += parser.statistics_output
message.message += "\n\n"
when BackupStatus::BROKEN
message.message +="Backup did not complete\n"
message.message += parser.statistics_output
message.message += "\n\n"
when BackupStatus::ERROR
message.message += "Error running parsing script\n"
message.message += parser.error
message.message += "\n\n"
end
end
if parsers.size == 0
message.subject = 'Backup(s) unknown'
message.message = 'Status unknown, did not get a valid log.'
elsif ok_backups == parsers.size
message.subject = 'Backup(s) OK'
else
message.subject = 'Backup(s) failed'
end
message.subject += " (#{ok_backups}/#{parsers.size})"
message.log = input
message.send
end
--------------------------
Config file:
-------------------------
cat /etc/parse-rdiff-backup.conf
message:
main_contact: [EMAIL PROTECTED]
cc_contact: [EMAIL PROTECTED]
parser:
statistics:
Direction: Starting mirror|increment\ operation (.*)
Backup started: ^StartTime [\d\.]+ \((.*)\)$
#EndTime: ^EndTime (.*)$
#ElapsedTime: ^ElapsedTime (.*)$
Source files: ^SourceFiles (.*)$
Source files size: ^SourceFileSize (.*)$
#MirrorFiles: ^MirrorFiles (.*)$
#MirrorFileSize: ^MirrorFileSize (.*)$
#NewFiles: ^NewFiles (.*)$
New files size: ^NewFileSize (.*)$
#DeletedFiles: ^DeletedFiles (.*)$
Deleted files size: ^DeletedFileSize (.*)$
#ChangedFiles: ^ChangedFiles (.*)$
#Changed files size: ^ChangedFileSize (.*)$
#ChangedMirrorSize: ^ChangedMirrorSize (.*)$
#IncrementFiles: ^IncrementFiles (.*)$
#IncrementFileSize: ^IncrementFileSize (.*)$
Destination size change: ^TotalDestinationSizeChange \d+ \((.*)\)$
Errors: ^Errors (.*)$
Script is licensed under the GNU GPLv2 and Author is Christian Marie
while working for Solutions First.
thanks
dave
Greg Freemyer wrote:
> All,
>
> I've looked at the examples at
> http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem
> to address automated nightly scripts and error handling.
>
> Are there some more complex examples available?
>
> === Details
> I haven't used rdiff-backup for a couple of years and I never did have
> it integrated into a nightly cron script for production use.
>
> My needs have changed and I want to give it another shot.
>
> I'm backing up to a local directory on a dedicated backup disk so it is
> easy enough to add "rdiff-backup /src /backup" to my backup script.
>
> But what about catching errors?
>
> Seems like I should be sending output to a log file, grepping thru it
> and e-mailing it to myself if anything goes wrong.
>
> I've looked at the examples at
> http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem
> to address this common need.
>
> Thanks
> Greg
> --
> Greg Freemyer
> The Norcross Group
> Forensics for the 21st Century
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> rdiff-backup-users mailing list at [email protected]
> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
--
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
_______________________________________________ rdiff-backup-users mailing list at [email protected] http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
