Re: serializable kafka producer pool

Bump... Looking for a Kafka Producer Object pool to use in Spark Streaming inside foreachPartition On Wed, Jan 14, 2015 at 8:40 PM, Josh J joshjd...@gmail.com wrote: Hi, Does anyone have a serializable kafka producer pool that uses the KafkaProducer.crateProducer() method? I'm trying to use

generate specific throughput load

Hi, Is there a way to generate a specified amount of throughput? I'm using the Stats class here https://github.com/apache/kafka/blob/7130da90a9ee9e6fb4beb2a2a6ab05c06c9bfac4/clients/src/main/java/org/apache/kafka/clients/tools/ProducerPerformance.java to measure the throughput. Though I need to

serializable kafka producer pool

Hi, Does anyone have a serializable kafka producer pool that uses the KafkaProducer.crateProducer() method? I'm trying to use the Spark borrow feature to cache the kafka producers. Thanks, Josh

Kafka producer iOS and android

hi Is it possible to have iOS and android to run the code needed for Kafka producers ? I want to have mobile clients connect to Kafka broker Thanks, Josh

Re: Kafka producer iOS and android

service for kafka https://github.com/mozilla-metrics/bagheera . This project used kafka 0.7 but you can see how its implemented. Hope that helps. -Harsha On Mon, Oct 20, 2014, at 08:45 AM, Josh J wrote: hi Is it possible to have iOS and android to run the code needed

Re: Kafka producer iOS and android

Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Mon, Oct 20, 2014 at 1:09 PM, Josh J joshjd...@gmail.com wrote: Thanks for the tip. I would like to avoid hand rolling any code if possible. For example, on Android I would like

read N items from topic

hi, How do I read N items from a topic? I also would like to do this for a consumer group, so that each consumer can specify an N number of tuples to read, and each consumer reads distinct tuples. Thanks, Josh

rate of producers writing to a topic?

Hi, Is there a way to monitor the rate of rates to a particular topic? I wish to monitor the frequency of incoming tuples in order to consume from the topic in particular ways depending on the incoming write throughput. Thanks, Josh

Re: rate of producers writing to a topic?

Interested in the total number of tuples written per millisecond per topic. On Tue, Oct 7, 2014 at 3:56 PM, Josh J joshjd...@gmail.com wrote: Hi, Is there a way to monitor the rate of rates to a particular topic? I wish to monitor the frequency of incoming tuples in order to consume from

Re: rate of producers writing to a topic?

, Neil Harkins nhark...@gmail.com wrote: On Tue, Oct 7, 2014 at 3:56 PM, Josh J joshjd...@gmail.com wrote: Is there a way to monitor the rate of rates to a particular topic? I wish to monitor the frequency of incoming tuples in order to consume from the topic in particular ways depending

kafka high level consumer - threads guaranteed to read a single partition?

Hi, For the kafka high level consumer, if I create exactly the number of threads as the number of partitions, is there a guarantee that each thread will be the only thread that reads from a particular partition? I'm following this example

Re: consumer read from specific partition

Simple Consumer. -- Sharninder On Thu, Aug 14, 2014 at 2:16 PM, Josh J joshjd...@gmail.com wrote: Hi, Suppose I have N partitions. I would like to have X different consumer threads ( X N) read from a specified set of partitions. How can I achieve this? Thanks

Re: consumer read from specific partition

Is it possible to modify and use the high level consumer so that I can ignore processing certain partitions? On Mon, Aug 18, 2014 at 5:07 PM, Sharninder sharnin...@gmail.com wrote: On Mon, Aug 18, 2014 at 7:27 PM, Josh J joshjd...@gmail.com wrote: You can see an example of using

Re: consumer read from specific partition

and discarding messages that aren’t relevant. The only other option I can think of if you are using the high-level API would be finer-grained topics. Best Regards, -Jonathan On Aug 18, 2014, at 9:14 AM, Josh J joshjd...@gmail.com wrote: Is it possible to modify and use the high level

consumer read from specific partition

Hi, Suppose I have N partitions. I would like to have X different consumer threads ( X N) read from a specified set of partitions. How can I achieve this? Thanks, Josh

Re: In the middle of a leadership election, there is currently no leader...

AM, Josh J joshjd...@gmail.com wrote: Hi, I'm running three kafka brokers. I receive the following when using the producer panic: kafka server: In the middle of a leadership election, there is currently no leader for this partition and hence it is unavailable for writes