04.11.2013 20:16, Rainer Gerhards:
Let's ignore for a moment that I say it is a bug and take a bit broader look at it: why do we actually need this capability? What are the use cases? I am asking because, at least in new engine, this will have noticable overhead. I also have seen some feature inside the engine that also looked "cool", but after 6 years nobody seems to have used them. I'd like to avoid introducing new such ones. Feedback appreciated.
For me, this feature (possibility to take a message from one ruleset and submit it for processing into another ruleset with it's own queue) could be useful to decouple different actions. If secondary queue just discards messages when it is full, then it will not interfere with primary queue. Also, secondary queue could have different thread settings, which affects performance.
Currently, we can attach a queue to a single action, but this approach cannot be applied to multiple actions at once (or it is possible with omruleset only). Therefore, we cannot asynchronously parse/modify a message and then output it to a file. I haven't tried omruleset, because it is declared obsolete.
In current configuration, it is something unnatural that queue is a property of an action or a ruleset. As I see it: queue is a separate entity. A message may be passed to a queue from input module or from another place (such as omruleset, better named omqueue in this case, or another new statement). Then, the message can be dequeued towards a ruleset, which may consist of one or more statements (actions).
Another rather strange property of current engine is possibility to attach asynchronous queue to a message modification action. Messages are passed by reference to the queue, which means that they will be modified asynchronously, and their content is unpredictable for both main and secondary queues. Then, after modification, the message is just discarded from the secondary queue. This asynchronous processing, nevertheless, makes sense for real output actions, when a message can be considered read-only.
Thus, there should be two ways to pass a message to the secondary queue: by value (as a copy), or by reference. It could be automated with copy-on-write, but this can lead to unneeded copying if the message is not used in main queue.
To summarize: current call behaviour makes perfect sense (because call implies return), but, to be consistent, ruleset objects should not contain queue as a property. Or, at least, there should be warning when the queue is not of "direct" type.
And then, possibility to resubmit a message into another queue is also helpful. It can be done externally (via omfwd/imudp, for example), but, for performance reasons, it is certainly better to do this internally. Maybe, omruleset is the right way to go, it only lacks possibility to pass the message by reference for better performance.
-- Pavel Levshin _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
