On Fri, 4 Oct 2013, Mayur Patil wrote:
Hi there,
Thanks for the reply. My answers are in-line. Please have a look:
can you explain a little more about what you are trying to do?
I don't want to logs multiple entries of same component at same
timestamp.
do the message contents need to be identical for you to eliminate them?
are you ever going to enable more precise timestampes? if so, are these messages
still going to be duplicates?
Why are you wanting to eliminate what you see as being duplicate logs?
when you say you want to eliminate duplicate logs, what do you consider a
duplicate?
In this link, http://fpaste.org/44286/
what I observe is the component euca-cc is producing logs of two things:
instances and nodes. The duplicate is referred from the
timestamp there are totally 3 repetitive entries of both *instances and
nodes.
*
* *I want to eliminate them.
why are there so many duplicates in the first place?
These are requests made by Cloud components to check state.
How are these logs produced/transferred?
They are produced from Eucalyptus cloud which I exported to remote
rSyslog server.
In accordance with timestamp I want to eliminate them.
Please correct if I am wrong,
Seeking for guidance,
Thanks !!
well, the issue is that these are probably not actually duplicates, they are
probably properly logging that the same thing has happened multiple times in
that second.
to eliminate duplicates entirely, you would need to post-process the logs.
rsyslog does have a capability that was introduced into syslog many, many years
ago to detect if the same message body has been sent repeatedly (no matter what
the timestamp) and instead of logging every message, it will log one or a few of
them and then a log entry that says 'message repeated XXX times'
but modern logging practice is to disable this module, because if something is
happening a lot, you probably want to know about it (especially if it's an
error), and it's _really_ hard to alert on 'message repeated XXX times'
intelegantly, whild it's trivial to have something generate an alert if
something happens more than 50 times in a minute for example.
Given that your logs only have a couple duplicates in a row, the 'message
repeated' thing isn't likely to help you much.
In your example, you have two log entries that each happen 4 times within one
second, but a given log message never happens more than twice in a row, so the
'message repeated' module would not reduce the number of lines you would see.
I think you would be better off looking at why you are getting what you consider
to be unnessasary logs and address that.
If these are the result of status reports, it may be that you have multiple
things testing your systems. If so, why?
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
