So to resume there's 2 remaining issues i would like to address : * disabling octet count causes that my condition based on appname to be ignored * i have a segfault when i enable rebindInterval
Could you please tell me what are the information you need to help me ? Thank you again, Smana ----- Mail original ----- De: [email protected] À: "rsyslog-users" <[email protected]> Envoyé: Jeudi 23 Juillet 2015 15:47:56 Objet: Re: [rsyslog] Load balancing issue Actually i found the parameter and i changed my configuration. input(type="imtcp" port="514" ruleset="forward" supportOctetCountedFraming="off") But it still doesn't work as expected. Indeed inside my ruleset i have this condition : if $programname startswith 'Myapp.' then { action( type="omrelp" ... When i disable octet-counting it seems that this condition is not reached. My log format look like that : 20150115003549 server Myapp.sometag {"response":{"status":206,"duration":1,"size":311557},"some_other": 4242} And i use loggen (from syslog-ng) for my benchs. Note: when i change the condition to if $fromhost-ip == '10.x.x.x' then { ... it seems that the condition is reached and i still have a segfault thanks, Smana ----- Mail original ----- De: "Rainer Gerhards" <[email protected]> À: "David Lang" <[email protected]> Cc: "rsyslog-users" <[email protected]> Envoyé: Jeudi 23 Juillet 2015 15:13:24 Objet: Re: [rsyslog] Load balancing issue 2015-07-23 15:12 GMT+02:00 David Lang <[email protected]>: > On Thu, 23 Jul 2015, [email protected] wrote: > >> 1- it solved my issue regarding the disk queue not created. i'll update >> the github issue. > > > good. > >> 2- i have a lot of errors like "rsyslogd: Framing Error in received TCP >> message: invalid octet count -1871509715. [v8.10.0]" > > > this means that you are getting malformed data sent to you. Rsyslog > implements an extension to the syslog protocol where instead of each log > message being a string of text followed by a newline, the sending system can > send a number at the beginning (instead of <PRI> where PRI is the combined > facility/severity data) and rsyslog will then read that number of bytes as > the message. This allows a message to contain embedded newlines. > > What's happening is that you have something sending you digits at the > beginning of the message, rsyslog is trying to interpret this, but it's > garbage data. I don't know if there is a way to disable octet counted mode > on the reciever or not. There is a parameter to do that, but I don't remember the name out of my head. Rainer > similarly, a message starting with 'z' is > interpreted as a compressed message. > > The 'best' answer is to figure out which system is generating the invalid > messages and fix it there. If you can do so. > > David Lang > >> 3- As soon as i enable the "rebindInterval" option, rsyslog segfaults >> [Thu Jul 23 12:46:03 2015] rs:analytics qu[19247]: segfault at 20 ip >> 00007f3a64efa624 sp 00007f3a5b1f5bc8 error 4 in >> librelp.so.0.1.0[7f3a64eee000+11000] >> >> Please find the startup debug logs here : >> https://gist.github.com/Smana/21f1add821b91f1a0bc1 >> >> Regards, >> Smana >> >> >> >> ----- Mail original ----- >> De: "Brian Knox" <[email protected]> >> À: "rsyslog-users" <[email protected]> >> Envoyé: Jeudi 23 Juillet 2015 14:17:05 >> Objet: Re: [rsyslog] Load balancing issue >> >> Aha! David - to summarize, is the problem then that: >> >> a) the parameter did not exist previously, and >> b) was only added for the new style configs? >> >> Brian >> >> On Thu, Jul 23, 2015 at 7:59 AM, David Lang <[email protected]> wrote: >> >>> On Thu, 23 Jul 2015, Brian Knox wrote: >>> >>> From your diagram, it looks like you are trying to load balance RELP. As >>>> >>>> far as I know, RELP does not suppot ActionTCPRebindInterval. I believe >>>> this has been discussed on the mailing list: >>>> >>>> http://lists.adiscon.net/pipermail/rsyslog/2013-May/032549.html >>>> >>>> Unless something has changed, you need to use the omfwd module if you >>>> want >>>> to use tcp rebinding. This isn't a bug - this is documented behavior. >>>> The >>>> rebind interval parameter is documented as a parameter for omfwd. RELP >>>> uses omrelp, which has no such paramater. See: >>>> http://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html >>>> and >>>> http://www.rsyslog.com/doc/v8-stable/configuration/modules/omrelp.html >>>> >>> >>> with the new style config it does in the current git branch. It looks >>> like >>> it was added in 7.3.15 >>> >>> /* tables for interfacing with the v6 config system */ >>> /* action (instance) parameters */ >>> static struct cnfparamdescr actpdescr[] = { >>> { "target", eCmdHdlrGetWord, 1 }, >>> { "tls", eCmdHdlrBinary, 0 }, >>> { "tls.compression", eCmdHdlrBinary, 0 }, >>> { "tls.prioritystring", eCmdHdlrString, 0 }, >>> { "tls.cacert", eCmdHdlrString, 0 }, >>> { "tls.mycert", eCmdHdlrString, 0 }, >>> { "tls.myprivkey", eCmdHdlrString, 0 }, >>> { "tls.authmode", eCmdHdlrString, 0 }, >>> { "tls.permittedpeer", eCmdHdlrArray, 0 }, >>> { "port", eCmdHdlrGetWord, 0 }, >>> { "rebindinterval", eCmdHdlrInt, 0 }, >>> { "windowsize", eCmdHdlrInt, 0 }, >>> { "timeout", eCmdHdlrInt, 0 }, >>> { "localclientip", eCmdHdlrGetWord, 0 }, >>> { "template", eCmdHdlrGetWord, 0 } >>> }; >>> >>> >>> >>> >>> I use ActionTCPRebindInterval with haproxy with plain TCP. It works >>> very >>>> >>>> well. >>>> >>>> Cheers, >>>> Brian >>>> >>>> On Thu, Jul 23, 2015 at 7:03 AM, <[email protected]> wrote: >>>> >>>> With the architecture enclosed. >>>>> >>>>> >>>>> ----- Mail original ----- >>>>> De: [email protected] >>>>> À: "rsyslog-users" <[email protected]> >>>>> Envoyé: Jeudi 23 Juillet 2015 11:59:35 >>>>> Objet: [rsyslog] Load balancing issue >>>>> >>>>> Hello all, >>>>> >>>>> I'm currently trying to load balance the log traffic accross several >>>>> servers. >>>>> I thought my configuration with "ActionSendTCPRebindInterval" option >>>>> was >>>>> working properly, unfortunately my recent benchs show that the log flow >>>>> is >>>>> not well load balanced. >>>>> >>>>> Please find below a part of the architecture : >>>>> >>>>> >>>>> >>>>> >>>>> My problem is located on the log aggregators : the rsyslog send its >>>>> traffic to haproxy on localhost using relp protocol. >>>>> I monitored the tcp sessions and i can see that haproxy doesn't change >>>>> the >>>>> destination servers. >>>>> >>>>> watch 'ss -lap -o state established \( dport = :20514 \)' >>>>> Recv-Q Send-Q Local Address:Port Peer Address:Port >>>>> 1716 0 127.0.0.1:43652 127.0.0.1:20514 users:(("rsyslogd",8409,88)) >>>>> 0 1138 10.17.252.4:58436 10.19.12.5:20514 timer:(on,196ms,0) >>>>> users:(("haproxy",3922 >>>>> ,2)) >>>>> 1760 0 127.0.0.1:43650 127.0.0.1:20514 users:(("rsyslogd",8409,22)) >>>>> 0 0 10.17.252.4:55583 10.19.12.6:20514 users:(("haproxy",3922,10)) >>>>> >>>>> Please find enclosed my configuration. >>>>> >>>>> NB : >>>>> - the source pid (rsyslog) never change as it is expected with >>>>> "ActionSendTCPRebindInterval" >>>>> - i mixed legacy and new syntaxe because of the following bug >>>>> https://github.com/rsyslog/rsyslog/issues/96 >>>>> This bug is annoying and i didn't receive any update since about 4 >>>>> months >>>>> >>>>> Could you please help me ? >>>>> >>>>> OS : debian7 >>>>> rsyslog version : 8.10 >>>>> >>>>> Regards , >>>>> Smana >>>>> >>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>>> _______________________________________________ >>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>> >>> >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
