Alright, here's what I have.
--
Justin Brehm
----- Original Message -----
From: "Frieder Kundel" <[EMAIL PROTECTED]>
To: [email protected]
Sent: Tuesday, April 29, 2008 7:38:12 AM (GMT-0500) America/New_York
Subject: Re: Nagios plugin?
the third check seems to be what I have in mind - currently I monitor the
databases with
check_mysql_query -H ... -u mog -p .. -d mogilefs -q "select 1"
Are you willing to share your checkscript?
/fk
2008/4/28 Justin H. Brehm < [EMAIL PROTECTED] >:
I've had three different checks that I've used and all seem to have flaws.
First one was a simple TCP port check on the ports that MogileFS has open. This
is cool if you want to make sure the daemons are still running, but I noticed
that there were cases when a DB could go down and the port remains open.
Next I wrote something that used 'mogtool' to test injections and extractions,
however 'mogtool' does way more than I needed it to do and it would also tend
to keep retrying in areas if mogile went down making the nagios plugin NRPE
timeout.
The last thing that I wrote was script that uses the MogileFS::Client perl
modules and does an injection, extraction and I then compares the in/out files
size to simply check if we have the same file. This is what we've been using so
far, however, I have seen an instance where the database was down and
MogileFS::Backend would have a return code of '82' or something in that range
and my nagios check was giving me the UNKNOWN status. That was a long night of
moving some development databases, so I wasn't up to debugging it that night
and haven't revisited yet.
What I'm planning on doing, because most of the problems that I've seen tend to
revolve around the database side, will be modifying my last nagios plugin to do
a 'select 1' query on the Mogile DB first and if that fails then to alert. At
least I'll elimnate that first and then move on to testing whether the trackers
are functioning, etc.
--
Justin Brehm
Systems Engineer
iContact.com
----- Original Message -----
From: "Frieder Kundel" < [EMAIL PROTECTED] >
To: [email protected]
Sent: Monday, April 28, 2008 10:18:42 AM (GMT-0500) America/New_York
Subject: Nagios plugin?
Hi folks,
how do you monitor your mogile? Has anyone written a nagios plugin?
Best regards,
Frieder Kundel
--
Frieder Kundel
#!/usr/bin/perl
# vim:ts=4 sw=4 ft=perl et:
use strict;
use warnings;
use Getopt::Long;
use LWP::UserAgent;
use LWP::Simple;
use MogileFS::Client;
# Written by Justin Brehm ([EMAIL PROTECTED]) - 05/07/2008 14:09:00
#
# check_mogile.pl - Nagios Plugin to check the functionality of MogileFS
# Requirements - mysql-client
my %ERRORS=('OK'=>0, 'WARNING'=>1, 'CRITICAL'=>2, 'UNKNOWN'=>3);
my %opts=('trackers'=> 'localhost:7001', 'domain'=>'default', 'dbhost'=> 'mysqldb001.example', 'db' => 'mogilefs', 'dbuser' => 'mogilefs', 'dbpass' => 'lolwut');
my $host = `uname -n`;
$host =~ s/[\r\n]//g;
my $test_key = "test_" . $host . "_" . time();
my $test_file = "/etc/mogilefs/mogilefs.conf"; # file that we are going to inject/extract
unless ( -e $test_file ) {
print "MOGILEFS UNKNOWN - File $test_file doesn't exist. Please create and try again\n";
exit($ERRORS{"UNKNOWN"});
}
my $mysql = '/usr/bin/mysql'; # specify path to mysql client binary
unless ( -X $mysql ) {
print "MOGILEFS UNKNOWN - $mysql isn't executible or doesn't exist\n";
exit($ERRORS{"UNKNOWN"});
}
abortWithUsage() unless
GetOptions(
't|trackers=s' => \$opts{trackers},
'd|domain=s' => \$opts{domain},
'host|dbhost=s' => \$opts{dbhost},
'db=s' => \$opts{db},
'u|dbuser=s' => \$opts{dbuser},
'p|dbpass=s' => \$opts{dbpass},
);
abortWithUsage() unless
($opts{trackers} && $opts{domain} && $opts{db} && $opts{dbuser} && $opts{dbpass} && $opts{dbhost});
# Verify that the MySQL database is available first
my $mysql_check = `$mysql --user=$opts{dbuser} --password=$opts{dbpass} --host=$opts{dbhost} $opts{db} -e 'select 1' 2>/dev/null`;
if (!$mysql_check) {
print "MOGILEFS CRITICAL - MogileFS database $opts{db} is unavailable\n";
exit($ERRORS{"CRITICAL"});
}
my $mogfs = get_mogfs();
my $in_bytes = inject();
my $out_bytes = extract();
cleanup(); # Delete the file from MogileFS
if ($in_bytes eq $out_bytes) {
print "MOGILEFS OK - Key $test_key inject/extract success\n";
exit($ERRORS{"OK"});
} else {
error("Store size: $in_bytes\nRetrieved size: $out_bytes");
}
sub abortWithUsage {
print "check_mogile.pl -- Checks the functionality of MogileFS for iContact\n\n";
print "Usage:\t\$ check_mogile.pl --trackers=127.0.0.1:7001 --domain=publisher [--dbhost=ral-sql002.colo] [--db=mogilefs] [--dbuser=icontact] -[-dbpass=secret]\n\n";
print "\ttrackers\t: MogileFS trackers and port (127.0.0.1:7001)\n";
print "\tdomain\t\t: MogileFS domain\n";
print "\tdbhost\t\t: Hostname/IP of MySQL database\n";
print "\tdb\t\t: The MogileFS database\n";
print "\tdbuser\t\t: User with SELECT privileges on database\n";
print "\tdbpass\t\t: password for user\n";
exit($ERRORS{"UNKNOWN"});
}
sub error {
my $err = shift() || "ERROR: no error message provided!";
my $mogerr = undef;
if ($mogerr = $mogfs->errstr) {
$mogerr =~ s/^\s+//;
$mogerr =~ s/\s+$//;
}
my $syserr = undef;
if ($@) {
$syserr = $@;
$syserr =~ s/[\r\n]+$//;
}
print "CRITICAL - ";
print "$err\n";
print "MogileFS backend error message: $mogerr\n" if $mogerr;
print "System error message: [EMAIL PROTECTED]" if $syserr;
exit($ERRORS{"CRITICAL"});
}
sub get_mogfs {
my @trackerinput = split(/\s*,\s*/, $opts{trackers});
my @trackers;
my %pref_ip;
foreach my $tracker (@trackerinput) {
if ($tracker =~ m!(.+)/(.+):(\d+)!) {
$pref_ip{$2} = $1;
push @trackers, "$2:$3";
} else {
push @trackers, $tracker;
}
}
my $handle = MogileFS::Client->new(
domain => $opts{domain},
hosts => [EMAIL PROTECTED],
timeout => 3,
)
or error("Could not initialize MogileFS");
$handle->set_pref_ip(\%pref_ip);
return $handle;
}
# Injects into the default class. If we want to inject to a specific class
# we'll have to modify this.
sub inject {
my $src = $test_file;
my $key = $test_key;
my $bytes = $mogfs->store_file($key, undef, $src)
or error("Injection test failed");
return $bytes;
}
# Only returns that file size that is being extracted. To get the full
# file you'd need to return $extractfile instead
sub extract {
my $key = $test_key;
my $file;
$file->{type} = 'file';
$file->{maxnum} = 1;
$file->{parts}->{1} = {
paths => [ grep { defined $_ } $mogfs->get_paths($key) ],
};
unless (scalar(@{$file->{parts}->{1}->{paths}})) {
error("Extraction test failed");
}
my $extractfile;
foreach my $path (@{$file->{parts}->{1}->{paths} || []}) {
$extractfile .= get($path);
}
my $size = length $extractfile;
return $size
}
sub cleanup {
my $key = $test_key;
unless ( $mogfs->delete($key) ) {
error("Cannot delete $key");
}
}
