Ok, so I've overlooked possibility to attach queue to a ruleset. Then,
the ruleset can contain multiple actions, which are executed
sequentially. And the ruleset can be called from main queue. Therefore,
I can separate normalization under it's own queue with distincs output.
Nevertheless, in my previous configuration it was like this:
Ruleset(name="logd"){
Action(
queue.workerthreads="1"
queue.type="FixedArray"
queue.size="1" # this is to
minimize processing delay
queue.timeoutenqueue="0"
type="mmnormalize"
userawmsg="on"
rulebase="/etc/rsyslog.d/rules.rb"
)
if $!date != "" then {
Action(
type="omfile"
DynaFile="normFilename"
template="normTemplate"
)
}
}
And there was some reasonable output. This means the normalization queue
works not on it's own copy of a message, but on the original message.
So, there is always possibility for race condition.
Current config, just for reference:
Ruleset(
name="logd"
queue.workerthreads="5"
queue.size="100000"
queue.timeoutenqueue="0"
){
Action(
type="mmnormalize"
userawmsg="on"
rulebase="/etc/rsyslog.d/rules.rb"
)
if $!date != "" then {
Action(
type="omfile"
DynaFile="normFilename"
template="normTemplate"
)
}
}
--
Pavel Levshin
19.10.2013 19:23, Pavel Levshin:
If it was true, I would expect to never see normalized messages. But
they are there from time to time.
But if it is true, then queues are counterproductive for message
modification. I think, if so, queueing should be prohibited in
software for such modules. And then, all CPU intensive processing
would be done from main queue only?
--
Pavel Levshin
19.10.2013 19:05, David Lang:
Second, in 7.5.5, this does not work at all. In 7.4.4, mmnormalize
produces expected output, though it cannot keep the load and has
known memory leak. On the other hand, in 7.5.5 it produces rare
records in the output file (say, 100 per minute). It looks like it
cannot match other records, but this is weird, as they are all in
the same format and happily match in lognormalize.
Maybe I've just found why is this. Different output modules seem to
be called in parallel in 7.5.5, and mmnormalize adopts output module
interface. Therefore, if mmnormalize is faster than omfile, then
omfile can see results of mmnormalize. But it is very unlikely.
Am I right? Is parser module interface better for message
modification modules?
actions take place in the order they are listed in the config file,
unless you define queues for the different actions, in which case
they just get copies to the queues, then each queue operates
independantly.
so if you put mmnormalize in something with it's own queue, only that
queue will see the modified message
There should never be a race condition between modules.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
