On Fri, 25 Jun 2010, Rainer Gerhards wrote:
Reply to bath David's and Aaron's post:
-----Original Message-----
From: [email protected] [mailto:rsyslog-
[email protected]] On Behalf Of Aaron Wiebe
Sent: Friday, June 25, 2010 5:36 PM
To: rsyslog-users
Subject: Re: [rsyslog] RainerScript revited? NEW rsyslog.conf format
I agree, we're starting to get to the point of wondering why again.
So here is my reason why:
My basic concern is scoping. A lot of the current format is
order-specific, implicitly scoped configuration that makes it
extremely hard to work with. With more explicit scoping, I'm happy.
... that's one reason for the config language.
Next I see a large amount of questions that boil down to the missing
if/then/else functionality (as elaborated in another post).
The functionality to use a single action in more than one rule is missing
(but can be emulated using omruleset).
Config-reload on HUP with the current system is impossible (not a problem of
the config language, but of the engine). Same for full support for dropping
privileges.
Probably a handful of other, minor things (one being the somewhat annoying,
always-heard complaint that the config language is hard to work with, ugly,
unintuitive, ...).
I would propose the following (more or less in order of difficulty)
introduce scoping
whenever you see a "{" in the config file, save the current config
options to a stack. when you see a "}" pop the config options from the
stack (undoing any changes in between)
introduce statement blocks
change the parser so that wherever it currently allows one action make
it accept a {} block of actions (treat them as if they were multiple
actions with & as the selector)
introduce named actions
something like sub NAME <action> to define and use NAME to use
introduce if-then-else
internally you could convert it to
ruleset {
if condition {
block
discard}
{ block }
}
introduct the ability to implcitly define a ruleset
if an action is a condition (i.e. nested configutations) then implicitly
create a new ruleset to resolve the nesting.
with these capabilities available I think that this will allow for
straightforward config files representing very complex configurations with
very little change internally to rsyslog.
I suspect that the result is going to have some bottlenecks in complex
configurations due to all the different rulesets and the passing of
messages between them, but once the capability is in place in the config
file the implementation under the covers could change later to be better
optimized.
as far as the rsyslog config being hard to understand, I think there are
two conditions here.
1. very simple configs where the fact that rsyslog can use a traditional
syslog config file (with a header on it that seldom needs to change) is a
huge advantage
2. complex configs where the inability to define scope and nest things is
a major problem and makes the resulting config hard to write.
David Lang
But, after all, I managed to use the same config language for years now. When
Adiscon considered changing its overall config language (probably going XML
now, but not yet decided), I thought this was a nice chance to get some
momentum for a new rsyslog config system.
However, if the practical requirement boils down to scope (and it looks that
way), I am more than happy. All I need to do in that case is change all
config values to auto-reset after change. That means more config code must be
written (copied), but you always know what is active. Probably a good week of
work to do (plus a switch to enable the old behavior for compatibility). I
have no problem with that and could continue to look into some "more
interesting" things (from my POV, like performance).
And I know this may not be a popular idea, but really, I think it
should be considered -- syslog-ng's format is actually quite
straightforward and easily understood. Having a format similar may be
a problem for "product differentiation", but honestly, the syslog-ng
method of inputs, outputs, and functions to tie those together is
quite appropriate for a logging application. I'm not saying we should
use their exact format, but I do like lifting some of their better
config ideas. (I'm also partial to the bind-style/syslog-ng-style
config format.)
The config language closely reflects the internal design. If I use syslog-ng
config, I tie myself into their concepts and restrict me to them. This is not
something I want to do.
Rainer
-Aaron
On Fri, Jun 25, 2010 at 11:29 AM, <[email protected]> wrote:
On Fri, 25 Jun 2010, Rainer Gerhards wrote:
After all formats seem to have at least some problems attached, I
decided to
have a look at my original RainerScript idea. Note that the current
(partial)
implementation is NOT suitable for very high performance. However,
that is
something that can be changed.
To understand why a language, and why a specialized one, we need to
look at
rsyslogd from a different angle: to me, rsyslogd is high performance
processor good at shuffling messages from an input to an output and
applying
transformations while doing so. There is conditional logic involved
to craft
the path a specific message takes.
So one can think of rsyslog like a specialized computer whom's
instruction
set is optimized for that purpose (I guess that is what David was
referring
to when he talked about "functions").
With that view, the configuration file is actually a programming
language:
one that specifies how to process message. Reading the config file
can then
be compared to a typical compilation, where the generated object
code is
actually targeted towards "rsyslog machine instructions".
Obviously, to program that very special machine, we do need a
specialized
programming language as well. Alternatively, one may think of this
language
as being the "assembly language" of rsyslog, which closely matches
the
instruction set and thus can be used to write very efficient
programs (but
requires some skills for non-trivial cases).
Having said this, all language constructs must translate natively to
rsyslog
engine objects. At the same time, they should be very simple to use
for
novice users while providing all the expressiveness an expert user
needs.
This most importantly means that the config language must be easy to
read and
edit by a human.
[Michael: *this* actually are the requirements for the config
language,
together with the other long post this morning -- I am not sure if I
find
time to consolidate these two today.]
Now please have a look at a new sample config:
http://download.rsyslog.com/rainerscript_rsyslog.conf
This time, I have not only included a hypothetical complex
configuration, but
also a very simple standard config - just so that we can see how
verbose it
gets. All configs are inside a single file -- you need to scroll
down.
All in all, I have to admit I begin to like the RainerScript
approach once
again. Unfortunately, it obviously is the solution that requires
more work
than any others (because I need to craft the grammar and the parser
myself).
after reading this I am wondering why everything needs to change.
what concepts are missing in the current config language that you are
needing to add
as I see it you need to add
ability to define blocks of actions
currently this is done by changing the ruleset, this needs to be
made
more explicit and obvious
a block of actions may be a single action that's used more than
once
ability to define if-then-else not just if-then
combining these two to make the if-then or if-then-else do blocks of
actions.
what else is missing?
if you were going to a standard config language that other things
could
understand then changing everything to fit that language makes sense,
but
if it's going to be another custom config parser what's the benifit
of
starting from scratch rather than just adding the features to the
existing
one (since you will have to maintain the existing one anyway)
David Lang
P.S. is the rest of adiscon going to just go with XML?
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
