I suggest that you open a github issue tracker with as many details as possible (e.g. which error messages?) . That makes it easier to ensure it is adressed as part of the current kafka work. We have a kafka project defined on github and keep track of to-do items via it.
Rainer 2017-05-30 21:50 GMT+02:00 Xavier Lange <[email protected]>: > I am using imrelp/omkafka to feed a kafka cluster and I find there are > some instabilities in my dynamic DNS environment. DNS is managed by > Kubernetes. > > I list all 7 of my brokers in the omkafka configuration and if any one > of those hosts goes down and the DNS name disappears, the omkafka > plugin spews errors, in a tight loop, about the host being down. > > Here's the config: > > action( > Broker=["kafka-0.kafka-headless.default.svc.cluster.local:9092", > "kafka-1.kafka-headless.default.svc.cluster.local:9092", > "kafka-2.kafka-headless.default.svc.cluster.local:9092"] > Type="omkafka" > Topic="the-topic" > confParam=["compression.codec=gzip", > "queue.buffering.max.messages=10000000"] > Template="the-topic-json" > errorfile="/dev/null" > Partitions.auto="on" > queue.type="linkedlist" > queue.size="100000" > queue.timeoutenqueue="0" > ) > > So if kafka-0, kafka-1, or kafka-2 disappear, then rsyslog can > overflow my docker logs with error messages. > > Is this type of behavior going to be addressed in the upcoming rsyslog > work? Is this something I can help with? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
