I am using imrelp/omkafka to feed a kafka cluster and I find there are
some instabilities in my dynamic DNS environment. DNS is managed by
Kubernetes.
I list all 7 of my brokers in the omkafka configuration and if any one
of those hosts goes down and the DNS name disappears, the omkafka
plugin spews errors, in a tight loop, about the host being down.
Here's the config:
action(
Broker=["kafka-0.kafka-headless.default.svc.cluster.local:9092",
"kafka-1.kafka-headless.default.svc.cluster.local:9092",
"kafka-2.kafka-headless.default.svc.cluster.local:9092"]
Type="omkafka"
Topic="the-topic"
confParam=["compression.codec=gzip",
"queue.buffering.max.messages=10000000"]
Template="the-topic-json"
errorfile="/dev/null"
Partitions.auto="on"
queue.type="linkedlist"
queue.size="100000"
queue.timeoutenqueue="0"
)
So if kafka-0, kafka-1, or kafka-2 disappear, then rsyslog can
overflow my docker logs with error messages.
Is this type of behavior going to be addressed in the upcoming rsyslog
work? Is this something I can help with?
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
