One more thing: performance impact is rather minimal. I'm using mmjsonparse to look after CEE-enhanced syslogs and I didn't notice a significant performance penalty. Anyway, for 5-10k messages/sec even my laptop holds without breaking a sweat.
2013/1/15 Radu Gheorghe <[email protected]> > Hi Xavier, > > 2013/1/15 Xavier Fustero <[email protected]> > >> Hi, >> >> I think what Radu suggested was to add the *mydirector1* out of the >> message >> log itself on the client side > > >> > @cee: {"directory": "mydirectory1", "actual_message": "this is a test >> > message"} >> > > Right! > > >> >> and use this field to create dynamic files on the server side which >> wouldn't affect the contains of the log itself. >> > > Yes and no. I mean, on the server side, you can just write > "actual_message" field as the message part of the log. So the log from the > application, which you "enriched" in the client's rsyslog config. > > So you don't modify the message as it was generated by the application, > but on the server side you can choose which parts of the message sent by > the client will be written to the file. > > Does that make sense? > > Best regards, > Radu > > >> >> Is this right Radu? >> >> Thanks a log, >> Xavi >> >> >> On 15 January 2013 11:17, Xavier Fustero <[email protected]> wrote: >> >> > Hi, >> > >> > answer inline >> > >> > On 15 January 2013 11:11, Rainer Gerhards <[email protected] >> >wrote: >> > >> >> >> >> >> >> > -----Original Message----- >> >> > From: [email protected] [mailto:rsyslog- >> >> > [email protected]] On Behalf Of Xavier Fustero >> >> > Sent: Tuesday, January 15, 2013 11:06 AM >> >> > To: rsyslog-users >> >> > Subject: Re: [rsyslog] Substract string from message >> >> > >> >> > Hi Rainer,Radu, >> >> > >> >> > thanks for your answer. I really appreciate both. >> >> > >> >> > The mmjsonparse example looks very interesting but after Rainer >> email I >> >> am >> >> > afraid about performance impact. We have several rsyslog servers on >> the >> >> > cloud and some are pretty busy. >> >> >> >> If what Radu posted fits your need, performance is not too much >> affected. >> >> I thought you wanted to actually remove a part of the message. That >> would >> >> required setting and modifying a number of local variables, which >> would be >> >> performance intense. >> >> >> > >> > Actually you are right. I want to remove part of the message. I thought >> > Radu solution allow that (read everything too quick...). >> > >> > Xavi >> > >> > >> >> >> >> Rainer >> >> > >> >> > Anyway, thanks a lot for your answers. I will let manager decide on >> >> which >> >> > direction should we move. >> >> > >> >> > Kind regards, >> >> > Xavi >> >> > >> >> > On 15 January 2013 10:55, Radu Gheorghe <[email protected]> >> >> > wrote: >> >> > >> >> > > Hi Xavier, >> >> > > >> >> > > 2013/1/15 Xavier Fustero <[email protected]> >> >> > > >> >> > > > Hi Radu, >> >> > > > >> >> > > > thanks for replying. >> >> > > > >> >> > > > Option 1 doesn't suitable for me as the strings will have >> different >> >> > > length. >> >> > > > >> >> > > > Regarding option 2 (regular expressions) I tested it and I could >> use >> >> > > > it >> >> > > to >> >> > > > create dynamic files like I am doing currently using msg:F,58:1. >> >> > > However, I >> >> > > > can't see how to use it to remove *mydirectory* string from the >> >> > > > original message sent by my clients and write this modified >> message >> >> > > > to the log >> >> > > file. >> >> > > > >> >> > > > Option 3 I should upgrade my current rsyslog version. It is >> planned >> >> > > > in >> >> > > very >> >> > > > close future sprints. Looked at it quickly but not sure 100% if >> it >> >> > > enables >> >> > > > me to do this. >> >> > > > >> >> > > > I have read that version 7 offers structured logs. Does anyone >> know >> >> > > > if >> >> > > this >> >> > > > enables you to remove some pieces of the original message like >> the >> >> > > > one I want to? If so, is there any good example? >> >> > > > >> >> > > >> >> > > Yes, so from the "sender" machine, you can make your output >> template >> >> > > write something like this for %message%: >> >> > > >> >> > > @cee: {"directory": "mydirectory1", "actual_message": "this is a >> test >> >> > > message"} >> >> > > >> >> > > Then on the "receiver" machine, with rsyslog 7 you can use >> mmjsonparse >> >> > > to parse this JSON and use the fields in templates. Here's a good >> >> resource: >> >> > > http://www.rsyslog.com/receiving-cee-enhanced-syslog-in-rsyslog/ >> >> > > >> >> > > So once you parse the logs, with the example above you can use the >> >> > > variables %$!directory% and %$!actual_message% in your templates. >> If >> >> > > you need to output all the JSON (without the @cee: cookie), use >> >> %$!all- >> >> > json%. >> >> > > >> >> > > >> >> > > > >> >> > > > Thanks a log, >> >> > > > >> >> > > >> >> > > Nice wordplay :) You're welcome :) >> >> > > >> >> > > Best regards, >> >> > > Radu >> >> > > _______________________________________________ >> >> > > rsyslog mailing list >> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > > http://www.rsyslog.com/professional-services/ >> >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE >> >> > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad of >> >> > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> > > DON'T LIKE THAT. >> >> > > >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com/professional-services/ >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE >> WELL: >> >> > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites >> >> beyond >> >> > our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE >> THAT. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> >> >> > >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
