I assist with a project that pretty heavily depends on liblognorm called "Sagan" (http://sagan.io).
While we have other "normalization" methods, we prefer liblognorm. Our community rulebase file is at: https://github.com/beave/sagan-rules/blob/master/normalization.rulebase I agree with David, we don't want 10 different ways to normalize a Cisco log. At the same time, Cisco logs sometimes differ just enough that you _might_ need multiple ways to normalize them. We have talked about "market place" for rule normalization for years now. It was always my impression that this would be part of the rsyslog team efforts. It sounds like you have enough on your plate, keeping track for rulebase isn't high on priority. I understand this. With Sagan, we are doing this "anyways". That is, we are creating rulebases for different types of logs either way. We commit them to the Sagan repo right now. I'd like to suggest the following for response: 1. Split off the "normalization.rules" base from Sagan and great a new, separate github repo for it. 2. If someone would like to add some rulebase "rules", they can do a "pull" request. 3. All rulebase "rules" need to have an example, anonymized log sample. Used for testing. 4. If the rules look good, then they can be merged. I'm certainly not trying to step on Brian's or anyone elses toe's. IMHO, Sagan will benefit from a project like this. Obviously, rsyslog will as well. This would likely bring other people outside rsyslog to the project as well). Let me know your thoughts and thank you. ----- Original Message ----- From: "Ryan Ward" <[email protected]> To: "rsyslog-users" <[email protected]> Sent: Thursday, June 23, 2016 8:51:48 AM Subject: Re: [rsyslog] mmnormalize rule database Re: mmgrok packages All as a newbie to rsyslog I think this is a great idea and would find a marketplace for rulebases and examples very beneficial. On Thu, Jun 23, 2016 at 7:06 AM, Brian Knox <[email protected]> wrote: > David - I'm sure I could get some time to devote to shepherding this, and I > could get some time and resources from our community team to write some > articles / tutorials about rsyslog + mmnormalize and generate some > publicity for the project. Additionally I have access to a decently large > sampling of logs from a reasonably scaled environment for testing. > > If this is something people are interested in and the only blocker is time > and resources let me talk to a couple of people today and I'll update the > list. > > Cheers, > Brian > > On Wed, Jun 22, 2016 at 7:24 PM David Lang <[email protected]> wrote: > > > On Wed, 22 Jun 2016, Joe Blow wrote: > > > > > What about soliciting people to start sharing their mmnormalize rules? > > > I've already shared my checkpoint rules, I could see about sharing my > > Cisco > > > rules as well. I avoid regex engines like the plague (for obvious > > > reasons), but would also like to see larger log source parsers adopted > > and > > > open sourced. > > > > > > Thoughts? Should we try and start a larger repository for parsing well > > > adopted log sources via liblognorm? > > > > This thought keeps getting raised. Yes this shoudl be done. The problem > is > > that > > nobody has stepped up to organize this. > > > > We don't want to have 50 different ways to handle the same Cisco message, > > but > > how do we pick which of the many different versions we are going to use? > > > > David Lang > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
