--On Sunday, August 25, 2002 10:38 PM -0400 Daniel J Urist
<[EMAIL PROTECTED]> wrote:
>
>> We then dump the contents of the database into a flat XML representation,
>> and use XSL stylesheets to translate the data into a mon.cfg file.
>
> But if you have the configuration in the database, why not hack the mon
> scheduler to read the config directly from the database instead of dumping
> the database, recreating the mon config file, and telling mon to go and
> re-parse it? This would be much more efficient, though it would require
> hacking the scheduler.
>
For several reasons.
First the database format is quite complex, as it provides some
functionality that doesn't exist in mon.cfg syntax. Such as allowing you
to define a default set of values for the various fields of a service TYPE,
but override some or all of them on a specific hostgroup/service pair.
(I.e. by default all 'ping' tests may run every 5 minutes, but on certain
core services you run them every 30 seconds. I give the 'ping' service
type an interval of 5 minutes, but on core-routers:ping I override that
with a 30 second interval. Then if later I want to change the ping tests
to be every 2 minutes on most hostgroups, I just have to change it in one
place.) Also the database format provides some amount of access control.
So the mail system administrator could change the intervals (etc) of the
tests on the mail servers, but not change the default values for those
tests, or the settings of other hostgroups. MON doesn't need to care about
that information.
Second, using XML has several advantages. We can have a XML Schema which
defines what data is required in various places, and use it to validate the
XML output, so we know when something is wrong and we're generating bad
data. It also allows us to put more data in the system then just what MON
cares (directly) about. Such as the host table isn't just a list of
hostnames, but also IP addresses. This allows us to generate an /etc/hosts
file which will always contain all the hosts we're monitoring. (So DNS
problems don't cause all monitoring to start failing.) We're also
intending to put information in the database which will be used to generate
config files for Cricket, so we can control our statistics gathering from
the same configuration source.
Finally NetSage is intended as an add-on to MON. I don't want to require
all sites that use MON to run a database server just for monitoring
purposes. Many people would prefer to just maintain a flat config file and
edit it by hand. I *am* considering writing patches to MON to allow it to
read an XML based config file directly, as I think am XML config file would
have advantages, like being able to use an XML viewer to read it easily,
and using an XML editor to make changes but still guarantee validity (by
validating against the XML Schema)) But that will be a non-trivial amount
of work, so I haven't started on it yet. (Once that exists, using an XML
database to store the document might give you what you want.)
> I think this is where modularizing the mon data structure and storing it
> with some sort of (hopefully transparent) object persistence mechanism
> would be a a big win, though it would require hacking the scheduler. This
> could give us database independence (via DBI; even CSV text files would
> be supported as a back end, so no need to install an RDBMS for small
> installations), and a consistent API to the configuration data that could
> be used by the mon scheduler and any configuration program. We could keep
> using the mon.cfg file as a configuration mechanism by taking out the
> parser code from the scheduler and parsing the file into objects instead
> of a monolithic data structure.
I think you're trying to abuse the perl module concept to adapt it to your
desires. In general I think there are only a few reasons for separating
perl code into separate modules. The most common is separating by
functionality. Either different pieces of code which *can* operate
together but do not *need to* (Date::Parse and Date::Format), or different
pieces of driver code of which only one is used at a time (DBI and DBD:*).
I don't think that applies to this case, as all of Mon::Config::* would
need to be used together.
Another reason is to make the code more managable, and easier to maintain.
For example the largest project I've worked on has been the CMU NetReg
project, (http://www.net.cmu.edu/netreg) which is 41000 lines of perl,
separated out over 40 different modules, in two separate trees (the
database itself and the web interface). Splitting it into multiple files
is necessary in order to be able to maintain it, and allow parallel
development between multiple developers. In the case of MON, a single file
is still reasonably maintainable, and since parallel development isn't
really an issue, multiple modules doesn't seem to make sense.
Code re-use is another good reason for modules. But here MON already is
using modules. Mon::Client provides all the functionality needed by any
MON client. Of the code in the main mon script, I don't really see much
that needs to be re-used elsewhere. The main server loop certainly only
needs to be in the server itself. Perhaps the config file parsing code
could go into its own module, so that an external program could also parse
the config file, but other then for providing a way to display all that
data in a 'pretty' fashion, I don't see many uses for external config
parsing. Clients don't need all that information (What they do care about
they can already get from the server via Mon::Client.)
On the other hand, I will admit that providing some amount of persistance
to the mon configuration data would have some advantages. In particular
I'd like to maintain more information across server reloads then just what
is currently disabled. We could continue the current practice of dumping
that data to a text file and reloading it on restart, but I think it would
be cleaner to use one of various data storage modules to dump the data to a
file on disk. Then on restart/reload the server would just need to
carefully only load the data that is still relevant. i.e. reload the old
configuration data, parse the current config file to update the
configuration, and then dump any data which is no longer relevant (deleted
hostgroups or services). For example just using Storage.pm we could call
'store' on a 'save scheduler state' request, and 'retrieve' on a 'load
scheduler state' request. We would have to make some careful decisions
about what to load and what to ignore, but I think it would be useful, as
we would no longer have a situation where a mon server restart causes an
upalert to be dropped (something was failing before the restart, but starts
working before mon checks it again after the restart). Right now I'm
handling that by not automatically resetting mon while something is
failing, but that has problems when the failure is because of something
which necessitates a configuration change.
>
> But whatever-- if you have something that works, that's great.
>
Oh it definitely works. Right now I'm auto generating mon.cfg's and
auth.cf's for three Mon servers. Two slave servers that do most of the
monitoring, and one master that receives traps from the slaves and sends
alerts. The configuration database is up to almost 4000 rows of data, and
the 3 mon.cfg files total about 2400 lines. All that automatically
generated every 15 minutes, distributed to the servers via rsync, and
loaded automatically (if changes occurred, and (on the master) if nothing
is failing right now). Since the host and hostgroup data in our database
is all coming from an external data source (the Network Registration system
mentioned above) whenever new devices are registered, or new machines are
added to the various service groups, they automatically end up in the
monitoring system a few minutes later.
-David Nolan
Network Software Developer
Computing Services
Carnegie Mellon University
