No, sometime occured when rsyslog start. On Fri, Dec 11, 2015 at 11:36 AM, Tait Clarridge <[email protected]> wrote:
> Does the issue occur after it rsyslog been running for a long time? > > On Thu, Dec 10, 2015 at 10:33 PM, Sury Bu <[email protected]> wrote: > > CentOS release 6.3 (Final) 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 > 12:19:21 > > UTC 2012 x86_64 x86_64 x86_64 GNU/Linux > > > > The -23 to -1 partitions only appears at some unknown special situation, > > and at most time the rsyslog kafka module can work fine in my > environment. > > > > On Fri, Dec 11, 2015 at 11:04 AM, Tait Clarridge <[email protected]> > wrote: > > > >> This might be a case of integer overflow with the partition counter if > >> MAX_INT is reached. > >> > >> @Sury - what OS/arch are you using? > >> > >> On Thu, Dec 10, 2015 at 1:35 PM, David Lang <[email protected]> wrote: > >> > I don't know kafka, but when dealing with issues like this, I like to > >> write > >> > the data to a local file using the template that you use to send it to > >> the > >> > remote system, that way you can check to see if there is something > odd in > >> > the log messages that trigger issues. > >> > > >> > David Lang > >> > > >> > On Thu, 10 Dec 2015, Sury Bu wrote: > >> > > >> >> Date: Thu, 10 Dec 2015 17:52:11 +0800 > >> >> From: Sury Bu <[email protected]> > >> >> Reply-To: rsyslog-users <[email protected]> > >> >> To: [email protected] > >> >> Subject: [rsyslog] rsyslog omkafka define partition problem > >> >> > >> >> > >> >> Hi All, > >> >> > >> >> I use rsyslog as log collect tool in the ELK stacks, logs will send > to > >> >> kafka, rsyslog.conf configured as below: > >> >> > >> >> module(load="imfile" PollingInterval="5") > >> >> module(load="omkafka") > >> >> $WorkDirectory /var/lib/rsyslog > >> >> input(type="imfile" File="/usr/local/tomcat6/logs/xx/log4j.log" > >> Tag="xx:" > >> >> Severity="info" Facility="local0") > >> >> > >> >> local0.* action(type="omkafka" topic="xx" > >> >> broker="elkkafka1.example.com:9092 > >> >> ,elkkafka2.example.com:9092,elkkafka3.example.com.p.xyidc:9092" > >> >> partitions.number="24" > >> >> confParam=["compression.codec=snappy","socket.keepalive.enable=true"] > >> >> queue.spoolDirectory="/usr/local/xx" queue.filename="rsyslogq" > >> >> queue.saveonshutdown="on" queue.size="1000000" > queue.type="LinkedList" > >> >> queue.highwatermark="600000" queue.lowwatermark="20000" > >> >> queue.discardmark="800000" queue.maxfilesize="1g" > >> queue.maxdiskspace="10g" > >> >> action.ResumeInterval="10" action.ResumeRetryCount="-1" > >> >> action.reportSuspension="on" > action.reportSuspensionContinuation="on") > >> >> > >> >> Sometimes I can find these ERROR messages in the /var/log/messages, > >> >> partitions becomes from -1 to -23, while it should from 0 to 23, can > >> >> anyone > >> >> meat the same problem? > >> >> > >> >> Dec 10 10:56:01 xx1 rsyslogd-2359: action 'action 7' resumed (module > >> >> 'omkafka') [v8.14.0 try http://www.rsyslog.com/e/2359 ] > >> >> Dec 10 10:56:01 xx1 rsyslogd-2424: omkafka: Failed to produce to > topic > >> >> 'xx' > >> >> partition -4: Local: Unknown partition [v8.14.0 try > >> >> http://www.rsyslog.com/e/2424 ] > >> >> Dec 10 10:56:11 xx1 rsyslogd-2359: action 'action 7' resumed (module > >> >> 'omkafka') [v8.14.0 try http://www.rsyslog.com/e/2359 ] > >> >> Dec 10 10:56:11 xx1 rsyslogd-2424: omkafka: Failed to produce to > topic > >> >> 'xx' > >> >> partition -3: Local: Unknown partition [v8.14.0 try > >> >> http://www.rsyslog.com/e/2424 ] > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com/professional-services/ > >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards > >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > >> DON'T > >> >> LIKE THAT. > >> >> > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com/professional-services/ > >> > What's up with rsyslog? Follow https://twitter.com/rgerhards > >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >> of > >> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T > >> > LIKE THAT. > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >> DON'T LIKE THAT. > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
