so, what functions will mmglobal need?
mmcount currently will create a count for each different appname
mmsequence will allow you to create a variable and incriment it at will
(it sounds like these can probably be generalized and unified somehow)
we need the ability to set a value that will be added to the variable set for
future messages
As I see it, there are two parts to this.
1. the part that sets the variables/modifies counters
2. the part that populates the current message variables ($! or $.) with the
current values.
part 1 needs to be thread safe since there may be multiple worker threads
modifying things at once.
how accurate do the numbers in part 2 need to be? if you try to have all the
counters be atomically updated and reported, there will be a lot of overhead,
but since you have one thread populating things while others are changing them
it seems like you may not have a lot of choice?
mmcount avoids this problem because it is single-threaded and what it does is
fixed (it maintains counters based on appname), so a single call can maintain
the counters and populate the message variables correctly in all cases.
but mmsequence sounds like it allows for much more user control, some of which
will need to be conditional, so it may be invoked from within a condition (and
therefor be done within a worker thread)
any string variable setting is going to be done conditionally within a worker
thread.
as I'm trying to type this out and think through it, I'm wondering if the right
answer isn't to do something that very much resembles the 7.4.4 global
variables.
mmglobal is going to need to store it's data somewhere, have that somewhere be
the $/ namespace like we had in 7.4.4 instead of having to modify each message's
variable space with all the global variables. don't allow global variables to be
used in anything other than the right hand side of a set statement (if you care
about it, you set a message variable equal to the global variable, and then
refer to the message variable in your if or template statement). This should
avoid all the loop unrolling problems
Then require function calls to change anything in the global namespace
It sounds like we would need something similar to the following set of functions
globalset(var='$/varname' subvar='$property!or!string' value='string or
number');
globalinc(var='$/varname' subvar='$property!or!string' step='number default 1');
globaldec(var='$/varname' subvar='$property!or!string' step='number default 1');
this would let you set a string variable, set and use a counter, including
counting based on the contents of a message
so if you wanted counts per hostname you would do something like:
globalinc(var='$/hostcounts' subvar='$hostname')
which would create $/hostcounts!<hostname> variables as needed.
when you want to use these counts, you can
set $.myhostcounts = $/hostcounts;
and then use them in the message
Then you would need something like
globalReportReset(var='$/varname')
that you would use instead of $/hostcounts in the set above that would
atomically set $.myhostcounts and then destroy all the variables (the reset
portion)
The one other thing that I think would end up being needed, is some way to do
something to access a variable based on a property, something like:
$.myhostcounts!{$hostname}
returning the value for the hostnmae of the current message
David Lang
On Tue, 22 Oct 2013, David Lang wrote:
Date: Tue, 22 Oct 2013 04:30:44 -0700 (PDT)
From: David Lang <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] global variable use cases
On Tue, 22 Oct 2013, Rainer Gerhards wrote:
On Tue, Oct 22, 2013 at 1:09 PM, David Lang <[email protected]> wrote:
well, I wouldn't want to have the module store a counter into the $!
namespace, too much chance of conflicting with something from a message,
and if I've got several things stored (for different purposes), I don't
want them to all show up when I output $!
let the user decided. For this, we have local vars.
Ok, I thought you were saying that they would go into $!, not $.
as you show below, doing this for an explicit save is very stratightforward
(although why do you need both key and store_to? I would think either would
be enough to specify the variable name to use)
but for counting and sequences, especially for ones that count separately
based on a message property (such as appname), this seems harder.
although I guess when you are setting up the counter, you can specify a base
variable, and allow the module to create subvariables from there.
store it in $. or in some different namespace (say $/ ;-p )
for counting and sequence numbers this sounds reasonable.
but what are you thinking about for storing other values? I'm expecthing
that most of these other values will be strings.
What's the difference?
e.g. (just psedudocode)
action(type="globals" key="mykey" value="some string" store_to=".localvar")
it may be useful to implement this as function rather than action or even a
totally new object -- makes integrating with expressions much easier (for
outputs, you must always go through a template).
I think a function would be better, doing it as an action brings along a lot
of baggage to confuse people (separate queues for example)
But the 7.5.4 globals vars cannot do this, as you cannot attach semantics
to them that *force* to write to a local var -- this is the root problem.
well, since they were in a different namespace entirely, they didn't need to
write to a local var
One other issue that this does create is that there are now two classes of
message modification modules
1. modules that could have multiple copies running, needing only to
coordinate enough to not be modifying the same message
2. modules that are doing things across messages, so multiple copies get
really messy
I'm not sure that this is really a whole lot better, but I guess it's taking
it away from directly being exposed to the users, so less potential for
confusion.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
