Re: Issue with kafka-server-stop on RedHat7

The issue might be due to https://unix.stackexchange.com/questions/343353/ps-only-prints-up-to-4096-characters-of-any-processs-command-line I guess the issue is with kafka version >0.10.0. More details: https://github.com/apache/kafka/pull/2515 Regards, Ravi On Tue, May 9, 2017 at 12:01 PM,

Restrict consumers from connecting to Kafka cluster

Is it possible to restrict Kafka consumers from consuming from a given Kafka cluster? -- *Regards,* *Ravi*

Get topic level detail from new consumer group command

./bin/kafka-consumer-groups.sh --group batchprocessord_zero --bootstrap-server kafka-1-evilcorp.com:9092 --new-consumer --describe Running the above ConsumerGroupcommad will describe consumer for all the topics it's listening to. Is there any workaround to get *only topic level detail*? ​ --

How to build kafka jar with all dependencies?

I used .*/gradlew jarAll* but still scala libs are missing from the jar? ​It should be something ​very simple which I might be missing. Please let me know if anyone knows. -- *Regards,* *Ravi*

Kafka Consumer Behaviour

I was writing Kafka consumer and I have a query related to consumer processes. I have a consumer with groupId="testGroupId" and using the same groupId I consume from multiple topics say, "topic1" and "topic2". Also, assume "topic1" is already created on broker whereas "topic2" is not yet

Starting KafkaMetricsReporter from Kafka Consumer/Producer

In ProducerPerformance class we use CSVMetricsReporter for metrics reporting. Which I think is actually started with the help of below function: KafkaMetricsReporter.startReporters(verifiableProps) Similarly I wrote my own producer and I have a custom implementation of KafkaMetricsReporter. But

Re: Starting KafkaMetricsReporter from Kafka Consumer/Producer

Never mind. I found the issue. Thanks, Ravi On Fri, Oct 10, 2014 at 11:47 AM, ravi singh rrs120...@gmail.com wrote: In ProducerPerformance class we use CSVMetricsReporter for metrics reporting. Which I think is actually started with the help of below function

connect.timeout.ms porperty not present in kafka 8 producer config

Kafka 07 has following property for producer. connect.timeout.ms5000the maximum time spent bykafka.producer.SyncProducer trying to connect to the kafka broker. Once it elapses, the producer throws an ERROR and stops. But when i checked in Kafka 08 config , I couldn't find any such property. Is

Producer connection timing out

Even though I am able to ping to the broker machine from my producer machine , the producer is throwing below expcetion while connecting to broker. I wanted to increase time out for producer but couldnt find any parameter for that in kafka 8. Any idea whats wrong here? [2014-10-08 09:29:47,762]

Connections from kafka consumer

I have few questions regarding Kafka Consumer. In kafka properties we only mention the zookeeper ip's to connect to. But I assume the consumer also connects to Kafka broker for actually consuming the messages. We have firewall enabled on ports, so in order to connect from my consumer I need to

Re: Kafka consumer - Mbean for max lag

The MaxLag mbean is only valid for an active consumer. So while the consumer is actively running, it should be accurate. On Fri, Oct 3, 2014 at 4:21 AM, Shah, Devang1 devang1.s...@citi.com wrote: Hi, Referring to http://kafka.apache.org/documentation.html#java Number of messages the

Re: kafka producer performance test

It is available with Kafka package containing the source code. Download the package, build it and run the above command. Regards, Ravi On Wed, Oct 1, 2014 at 7:55 PM, Sa Li sal...@gmail.com wrote: Hi, All I built a 3-node kafka cluster, I want to make performance test, I found someone post

Re: BytesOutPerSec is more than BytesInPerSec.

are using a replication factor of two? As Steven said, the replicas also consume from the leader. So it's your consumer, plus the replica. On Thu, Sep 25, 2014 at 10:04:29PM -0700, ravi singh wrote: Thanks Steven. That answers the difference in Bytes in and bytes Out per sec. But I

BytesOutPerSec is more than BytesInPerSec.

I have set up my kafka broker with as single producer and consumer. When I am plotting the graph for all topic bytes in/out per sec i could see that value of BytesOutPerSec is more than BytesInPerSec. Is this correct? I confirmed that my consumer is consuming the messages only once. What could

Re: BytesOutPerSec is more than BytesInPerSec.

wrote: couldn't see your graph. but your replicator factor is 2. then replication traffic can be the explanation. basically, BytesOut will be 2x of BytesIn. On Thu, Sep 25, 2014 at 6:19 PM, ravi singh rrs120...@gmail.com wrote: I have set up my kafka broker with as single producer and consumer

Re: Intercept broker operation in Kafka

:51 PM, ravi singh rrs120...@gmail.com wrote: Thanks Guozhang/Neha for replies. Here's my use case: We use proprietary application logging in our apps. We are planning to use Kafka brokers in production , but apart from the logs which are already logged using log4j in kafka we want

Are Broker stats stored in zookeeper?

I want to use some vital broker stats into a different logging system. Can i read the broker specific data from zookeeper? -- *Regards,* *Ravi*

Re: Intercept broker operation in Kafka

protocol and hence provide all sorts of hooks and plugins on the brokers at the cost of performance. Could you elaborate more on your use case? There is probably another way to model your application using Kafka. Thanks, Neha On Sat, Jun 21, 2014 at 9:24 AM, ravi singh rrs120...@gmail.com wrote

Intercept broker operation in Kafka

How do I intercept Kakfa broker operation so that features such as security,logging,etc can be implemented as a pluggable filter. For example we have BrokerFilter class in ActiveMQ , Is there anything similar in Kafka? -- *Regards,* *Ravi*