Re: Kafka streams state directory - help

Thanks Matthias. Can you also please confirm the compatible versions of the client dependencies . Our broker version is 10.2.1 and when i updgrade the client library to 1.1.0, i am getting a issue with tests while starting the embedded cluster . Test dependencies are (kafka-stream.version is

Re: Monitoring Kafka

Thanks guys for the pointers ! On Sat, Apr 21, 2018 at 9:53 PM, Steve Jang wrote: > The following tool is really good: > https://github.com/yahoo/kafka-manager > > > On Sat, Apr 21, 2018 at 5:42 AM, Joris Meijer wrote: > > > You can do this without

Re: Monitoring Kafka

The following tool is really good: https://github.com/yahoo/kafka-manager On Sat, Apr 21, 2018 at 5:42 AM, Joris Meijer wrote: > You can do this without exposing the JMX port, e.g. by using a Prometheus > exporter as javaagent (https://github.com/prometheus/jmx_exporter). > >

Re: Monitoring Kafka

You can do this without exposing the JMX port, e.g. by using a Prometheus exporter as javaagent (https://github.com/prometheus/jmx_exporter). Metricsreporters, such as the one from Confluent, also don't require you to open ports, because metrics will be pushed out of the broker (

Re: Monitoring Kafka

Without JMX may be difficult.. why not install an agent and report to an external service like ELK or new Relic? That’s long standing industry pattern. Some reading.. and some tools in the readings.. these articles are opinionated towards the vendors that published them but its a starting

Monitoring Kafka

Hi, Is there a way to pull broker stats (like partitions its is managing, jvm info, state of the partitions, etc.) without using JMX. We are shipping kafka in a appliance and there are restrictions on the ports that are open for security reasons. Are there any known ways of monitoring the health

Re: Kafka streams state directory - help

You are hitting: https://issues.apache.org/jira/browse/KAFKA-6499 Was fixed in 1.1 release. Thus, you can just ignore the checkpoint file. There should be no issue with running on Kubernetes. Also, if there is no store (independent of disk based or in-memory) there will be no changelog topic.

Re: Broker cannot start switch to Java9 - weird file system issue ?

Il sab 21 apr 2018, 06:29 Ismael Juma ha scritto: > Hi Enrico, > > It is a real problem because it causes indexes to take a lot more disk > space upfront. The sparsity is an important if people over partition, for > example. > Got it. In production I saw no issue, maybe due

Kafka streams state directory - help

Hi, I am using kafka streams app connecting to confluent kafka cluster(10.2.1). Application is reading messages from a topic, performing a tranformation and pushing to output topic . There is no count or aggregation performed . Have following clarifications regarding state directory. *1)* Will