Hello, I use the following set up to send netflow data to a Kafka broker:
pmacct 1.6.0 librdkafka 0.8.6-1.1 (from latest Ubuntu Package) Kafka 0.10.0.1 Scala 2.11 I run in a Kubernetes/Google Container Engine setup. The pmacct forwards all packets to the Kafka broker (I checked this with tcpdump). And if I configure a topic which isn`t there already, Kafka adds the topic (which I can check in the log folders). So it seems that the connection to Kafka is established correct. But the events don't get into the topic log and are not receivable with the console-consumer. In the logs I found the same error messages as there: http://www.mail-archive.com/pmacct-discussion%40pmacct.net/msg03141.html ERROR ( default/kafka ): Connection failed to Kafka: p_kafka_check_outq_len() ERROR ( default/kafka ): Connection failed to Kafka: p_kafka_close() INFO ( default/kafka ): *** Purging cache - END (PID: 1141, QN: 0/2016, ET: 1) *** INFO ( default/kafka ): *** Purging cache - START (PID: 1145) *** ERROR ( default/kafka ): Connection failed to Kafka: p_kafka_check_outq_len() ERROR ( default/kafka ): Connection failed to Kafka: p_kafka_close() INFO ( default/kafka ): *** Purging cache - END (PID: 1145, QN: 0/2461, ET: 1) *** INFO ( default/kafka ): *** Purging cache - START (PID: 1149) *** ERROR ( default/kafka ): Connection failed to Kafka: p_kafka_check_outq_len() ERROR ( default/kafka ): Connection failed to Kafka: p_kafka_close() INFO ( default/kafka ): *** Purging cache - END (PID: 1149, QN: 0/2606, ET: 1) *** ... ... I also tweaked the refresh time up, without any success. Thanks for every pointer, help, advice, ...
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
