On Thu, 18 Dec 2008, Rainer Gerhards wrote: > On Fri, 2008-12-19 at 03:46 -0800, [email protected] wrote: >> On Thu, 18 Dec 2008, Rainer Gerhards wrote: >> >>> Hi David, >>> >>> On Thu, 2008-12-18 at 21:38 -0800, [email protected] wrote: >>>> with messages appearing out-of-order the 'last message repeated X times' >>>> is pretty useless. >>> >>> Not really. Please note that form the perspective of the output module, >>> messages do not appear out of order. The output module receives a stream >>> of messages, and the "last message repeated x times" logic works on that >>> stream. So no matter if messages are re-ordered by async queues (or UDP >>> or whatever), the "last ... times" correctly reflects the way things >>> were handed to the output module. >> >> but if the output module is then relaying to another system, that other >> system (or the transport between them) can also re-order the messages > > ah, ok, remote scenario. Of course, that can happen. But that's "just" > one use case where it is problematic. > >>> But I concur that this feature, in its current state, is very >>> questionable. I've talked about this on the mailing list quite some >>> time, I think there is also at least one blog post about it. >>> >>>> >>>> I think I remember reading about an option to disable this, but another >>>> work-around to the problem is to change the output so that it becomes >>>> 'last message repeated X times %msg%', so you can see (most, if not all) >>>> of the message being repeated in the line telling you that it was >>>> repeated. >>> >>> As I said, the message in front of this message is either another repeat >>> message or the message that is being repeated. So you can always trace >>> back to what was repeated (but it is painful). >>> >>> Given the feedback I received on this feature (use cases), it should >>> probably (and v4 would be an appropriate version to do so) be redisigned >>> to be a rate-limiting feature on the input side. That would pretty much >>> simplify code, too. >> >> but since things can be re-ordered after the input module is done with >> them (multiple threads pulling from the queue, or relaying) it is still >> useful for the 'message repeated' message to have more info about what it >> is referring to. > > That makes sense, at least for scenarios where it is expected behaviour. > For others, it would probably break analysers. So it needs to be > optional. And probably on a per-action basis. I'll see if it is > sufficiently simple, but the whole "last message repeated" thing is > broken anyhow (IMO), I don't like the idea to invest any more than maybe > an hour into that feature, especially in the light that the whole idea > must be replaced in the not so distant future...
it's definantly a pain for analysers (a _lot_ of them really only look at one line at a time), that's why I started tweaking this on sysklogd, and while in theory just disabling it is the right answer, the ability to take a flood of inbound messages and reduce them to a much smaller number can save your logging system when things go wrong I'll watch to see what happens. David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
