Re: How to log/analyze the consumer lag in kafka streaming application

Hi, Thanks for sharing the info. I am reading this document for more understanding: http://kafka.apache.org/documentation.html#monitoring Is there any special way I need to start my kafka cluster or streams application (or configure them) to report these metrics. I suppose both cluster and

Re: How to log/analyze the consumer lag in kafka streaming application

You should check out Kafka Streams Metrics (for upcoming 0.10.2 they are even more detailed). There is not a lot of documentation for 0.10.0 or 0.10.1, but it work the same way as for consumer/producer metric that are documented. -Matthias On 1/24/17 10:38 PM, Sachin Mittal wrote: > Hi All, >

Upgrade questions

I have a few questions regarding an upgrade that Im attempting to perform. - Currently we are running version 0.8.2_beta. - We are trying to upgrade to 10.1.1 - Our setup uses the following path /server/kafka where kafka is a symlink to kafka-{version} - I attempt to perform the upgrade as

Re: Kafka Multiple Consumer Group for Same Topic

Hi Senthil, You can just try it yourself, using kafka-consumer-perf-test.sh bundled with Apache Kafka. You can start 2 in parallel with different groups and see what happens if you tune some parameters. Joris On Wed, Jan 25, 2017, 08:38 Senthil Kumar wrote: > Thanks

Re: Reg: Kafka ACLS

Why would you want to use the benefit of ACLs without mutual SSL? I think that as soon as you decided you need ACLs, you also should add encryption. It's relatively easy to configure. On Thu, Jan 26, 2017, 05:35 Manikumar wrote: > Yes, we can use Kafka ACL's with

Kafka JDBC connector vs Sqoop

I'm looking for ways to bulk/incremental import from MySQL database to HDFS. Currently I got Sqoop that does the bulk import creating a Hive table. Wanted to know the pros/cons of using JDBC connector instead of Sqoop and are there any MySQL config changes expected (like binlog configuration in

Fwd: [DISCUSS] KIP-114: KTable materialization and improved semantics

Forwarding this thread to the users list too in case people would like to comment. It is also on the dev list. Thanks Eno > Begin forwarded message: > > From: "Matthias J. Sax" > Subject: Re: [DISCUSS] KIP-114: KTable materialization and improved semantics > Date: 24

broker randomly shuts down

I know I'm reviving an old thread but did the original poster ever find the cause of this issue and figure out what the fix was? I am running a cluster of 18 Kafka .9 brokers and three of them are having behaving exactly this way once a week. Pretty scary because they are doing a full resign

using kafka log compaction withour key

Hello, Is is possible to use log compaction without key? I think in that case buffer will contain only one line of data value? Is that correct? thanks