On Tue, 22 Oct 2013, Rainer Gerhards wrote:
On Tue, Oct 22, 2013 at 11:09 AM, David Lang <[email protected]> wrote:
On Tue, 22 Oct 2013, Rainer Gerhards wrote:
On Tue, Oct 22, 2013 at 10:23 AM, David Lang <[email protected]> wrote:
Most folks would consider this to be an error ;) I agree to Pavel that this
is so far from being what one expects, that even documenting how things work
will not really help getting away from that error-like feeling. Add the fact
that the result is non-deterministic as it depends on the current batch
size, I would also conclude that this is errornous *design*.
I probably was not clear enoug in my initial question: as I said yesterday
evening, I have now removed global variable support. It won't be available
in 7.5.6. My question thus was on the use cases that could guide us if
working toward a correct implementation makes any sense. This would require
a big design change.
the use case wasn't trying to be something as precise as you are talking
about, but rather a much corser ability that after you see message X show
up, start doing something different. This could be to write files to a
different location because a failover has happened and you are now the
active box, it could be that if a particular type of log message shows up
you want to log debug messages, or something else.
given that logs can be re-ordered by the network, by relays in the middle,
or by the results of batch processing, it was never going to be an exact
thing, so if a change only takes effect for the next batch, that's not
unreasonable.
yes, this is edging towards log correlation territory, but it's an
extremely handy way to be able to turn things on and off, or to cause logs
to be handled differently without requiring on-the-fly editing of config
files and a complete restart.
the log losses from a complete restart are massive compared to the global
variable impresision of only setting the variable on the next batch.
global variables are not that useful as something that you would set for
each message that you process, but if they are only occasionally changed,
they can be very useful and not a significant performance issue.
It's not the performance issue -- it's the "strong smell of incorrectness".
Remember that I must support that whole thing in the future. People will
continously ask why it "does not work", and some of them will
(even/hopefully/probably) have support contracts, so I am actually in
charge to deliver a solution. If I then just describe "it is as it is" that
will create very unsatisfied customers. So better do it right or do it not
at all (which is what I currently opt for, and my peers will kill me if I
knowingly introduce a feature that potentially creates great
user-desatisfaction).
by default the batch size is still 1 correct? it's only people who are scaleing
up who will be setting it higher.
I think that if you just document that setting global variables is racy, and as
such it's not suitable for accruate counting, only for changing rsyslog behavior
without a restart you should be good on the expectations front. the current
documentation leans heavily on the counting aspect of things, but that can be
changed. remove any suggestion of future atomic opertations and emphisise that
atomic operations are not possible.
being able to enable or disable e-mail messages, change what the destination
address is, change the filename or patch when the box becomes active are all
very useful items that I would hate to loose.
even the load balancing hack works 'well enough' once you accept that you are
balancing per batch rather than per message (even if you did balance per
message, you really have no idea how expensive a particulare message is going to
be, so you are not really balancing the work precisely, you are only doing so
statistically, and balancing per batch rather than per message is just as valid
statistically)
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
