Hi All, I use rsyslog as log collect tool in the ELK stacks, logs will send to kafka, rsyslog.conf configured as below:
module(load="imfile" PollingInterval="5") module(load="omkafka") $WorkDirectory /var/lib/rsyslog input(type="imfile" File="/usr/local/tomcat6/logs/xx/log4j.log" Tag="xx:" Severity="info" Facility="local0") local0.* action(type="omkafka" topic="xx" broker="elkkafka1.example.com:9092 ,elkkafka2.example.com:9092,elkkafka3.example.com.p.xyidc:9092" partitions.number="24" confParam=["compression.codec=snappy","socket.keepalive.enable=true"] queue.spoolDirectory="/usr/local/xx" queue.filename="rsyslogq" queue.saveonshutdown="on" queue.size="1000000" queue.type="LinkedList" queue.highwatermark="600000" queue.lowwatermark="20000" queue.discardmark="800000" queue.maxfilesize="1g" queue.maxdiskspace="10g" action.ResumeInterval="10" action.ResumeRetryCount="-1" action.reportSuspension="on" action.reportSuspensionContinuation="on") Sometimes I can find these ERROR messages in the /var/log/messages, partitions becomes from -1 to -23, while it should from 0 to 23, can anyone meat the same problem? Dec 10 10:56:01 xx1 rsyslogd-2359: action 'action 7' resumed (module 'omkafka') [v8.14.0 try http://www.rsyslog.com/e/2359 ] Dec 10 10:56:01 xx1 rsyslogd-2424: omkafka: Failed to produce to topic 'xx' partition -4: Local: Unknown partition [v8.14.0 try http://www.rsyslog.com/e/2424 ] Dec 10 10:56:11 xx1 rsyslogd-2359: action 'action 7' resumed (module 'omkafka') [v8.14.0 try http://www.rsyslog.com/e/2359 ] Dec 10 10:56:11 xx1 rsyslogd-2424: omkafka: Failed to produce to topic 'xx' partition -3: Local: Unknown partition [v8.14.0 try http://www.rsyslog.com/e/2424 ] _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
