kafka-web-console

Hi, all I am installing kafka-web-console on ubuntu server, when I sbt package it, it stuck on waiting for ivy.lock root@DO-mq-dev:/home/stuser/kafkaprj/kafka-web-console# sbt package Loading /usr/share/sbt/bin/sbt-launch-lib.bash [info] Loading project definition from /home/stuser/kafkaprj/kaf

Re: Is kafka suitable for our architecture?

On 10/10/2014 05:12 AM, Albert Vila wrote: > Some comments below. > > On 10 October 2014 11:19, cac...@gmail.com wrote: > >> Albert, you certainly can use Kafka (and it will probably work quite well) >> you'll just need to make sure your consumers are written to match the >> available options. I

Re: Starting KafkaMetricsReporter from Kafka Consumer/Producer

Never mind. I found the issue. Thanks, Ravi On Fri, Oct 10, 2014 at 11:47 AM, ravi singh wrote: > In ProducerPerformance class we use CSVMetricsReporter for metrics > reporting. > Which I think is actually started with the help of below function: > KafkaMetricsReporter.startReporters(verifiable

Re: Installation and Java Examples

Mohit, Kafka uses gradle to build the project, check the README.md under source dir for details on how to build and run unit tests. You can find consumer and producer api here http://kafka.apache.org/documentation.html and also more details on consumer http://kaf

Installation and Java Examples

I am new to Kafka and very little familiarity with Scala. I see that the build requires "sbt" tool, but do I also need to install Scala separately? Is there a detailed documentation on software requirements on the broker machine. I am also looking for 3 different types of java examples 1) Follow r

Starting KafkaMetricsReporter from Kafka Consumer/Producer

In ProducerPerformance class we use CSVMetricsReporter for metrics reporting. Which I think is actually started with the help of below function: KafkaMetricsReporter.startReporters(verifiableProps) Similarly I wrote my own producer and I have a custom implementation of KafkaMetricsReporter. But to

Re: Auto Purging Consumer Group Configuration [Especially Kafka Console Group]

There is a JIRA open for exactly this issue with patch available: KAFKA-559 On Fri, Oct 10, 2014 at 9:33 AM, Neha Narkhede wrote: > Bhavesh, > > All consumer groups with at least one alive consumer will have a > registration node for that consume

Re: mount of messages produced and those consumed are not equal

Hi Shuo, What are your producer and consumer config values? Guozhang On Fri, Oct 10, 2014 at 3:47 AM, Shuo Chen wrote: > Dear folks, >I have run some basic tests using kafka. I have started 7 brokers, > created a topic with 21 partitions and 3 replicas. > > 2 producers are started with

Re: Auto Purging Consumer Group Configuration [Especially Kafka Console Group]

Bhavesh, All consumer groups with at least one alive consumer will have a registration node for that consumer instance under /consumers//ids. Any consumer group that is long gone will have no registered consumer instance in zookeeper. The right way to clean up defunct consumer groups is to scan co

Re: including KAFKA-1555 in 0.8.2?

+1. On Thu, Oct 9, 2014 at 6:41 PM, Jun Rao wrote: > Hi, Everyone, > > I just committed KAFKA-1555 (min.isr support) to trunk. I felt that it's > probably useful to include it in the 0.8.2 release. Any objections? > > Thanks, > > Jun >

mount of messages produced and those consumed are not equal

Dear folks, I have run some basic tests using kafka. I have started 7 brokers, created a topic with 21 partitions and 3 replicas. 2 producers are started with 5 threads each and pushed 2419200 messages of 50 bytes in total. 3 consumers with 2 in one group and 1 in other are started to fetc

Re: Scala+Kafka puzzler

Oh, right, well spotted, I was looking in wrong branch. In 0.8.2 branch it's no longer private. Thanks! Kind regards, Stevo Slavic On Fri, Oct 10, 2014 at 12:21 PM, Sean Owen wrote: > It's private to package "kafka": > > https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/clus

Re: Is kafka suitable for our architecture?

Some comments below. On 10 October 2014 11:19, cac...@gmail.com wrote: > Albert, you certainly can use Kafka (and it will probably work quite well) > you'll just need to make sure your consumers are written to match the > available options. I think I may not have a good picture of what you need

Re: Scala+Kafka puzzler

It's private to package "kafka": https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/cluster/Broker.scala#L29 On Fri, Oct 10, 2014 at 10:47 AM, Stevo Slavić wrote: > Hello Apache Kafka community, > > Attached trivial Maven built project with Kafka code fails to compile, with > er

Re: Is kafka suitable for our architecture?

Albert, you certainly can use Kafka (and it will probably work quite well) you'll just need to make sure your consumers are written to match the available options. I think I may not have a good picture of what you need to do. Is it that you have a stream of documents coming in and then each documen