Re: open source projects based on kafka

Hi Li, You might take a look at Apache Samza. It's conceptually simple but powerful and makes heavy use of Kafka. Best, Roger Sent from my iPhone > On Sep 12, 2015, at 10:34 PM, Li Tao wrote: > > Hi Hackers, > > This is Lee, a learner of kafka, i have read the

Re: open source projects based on kafka

Here's a toy project - analyzing twitter stream. 1) Create dev. account on twitter 2) Using your dev credentials, connect to twitter stream api to retrieve stream of tweets 3) Store tweets in Kafka (using Kafka producer) 4) Retrieve tweets (using Kafka consumer) 5) For each tweet (or group of

Re: open source projects based on kafka

Hi Roger, Thanks for your recommendation. I just got to know Samza. and checked its code base. It is a little too huge for me. Maybe for now, I need to start a small project/application which utilize kafka as its infrastructure, so that I can use Kafka's API a lot and know Kafka better. It's

Re: Broker restart, new producer and snappy in 0.8.2.1

Hi, the 0.8.2.2 release (which vote just passed and should be announced soon) has a patch that may be related https://issues.apache.org/jira/browse/KAFKA-2308 not sure. Here are the 0.8.2.2 artifacts https://people.apache.org/~junrao/kafka-0.8.2.2-candidate1/ I don't see them yet in Maven central

Re: open source projects based on kafka

The toy project idea is good. Another option I think could be to look at the various Kafka client langague bindings and/or utilities (like kafkacat). And from there, another option is to build a client language binding for a language that's kind of lacking Kafka support, some have better support

Re: open source projects based on kafka

Have a look at https://github.com/allegro/hermes On Mon, Sep 14, 2015, 01:28 David Luu wrote: > The toy project idea is good. Another option I think could be to look at > the various Kafka client langague bindings and/or utilities (like > kafkacat). And from there, another

Re: open source projects based on kafka

You could try implementing your own client. https://cwiki.apache.org/confluence/display/KAFKA/Clients On Sun, Sep 13, 2015 at 7:46 PM, Stevo Slavić wrote: > Have a look at https://github.com/allegro/hermes > > On Mon, Sep 14, 2015, 01:28 David Luu wrote:

Re: port already in use error when trying to add topic

Changing the port to 9998 did not help. Still the same error occurred On Sat, Sep 12, 2015 at 12:27 AM, Foo Lim wrote: > Try throwing > > JMX_PORT=9998 > > In front of the command. Anything other than 9994 > > Foo > > On Friday, September 11, 2015, allen chan

Re: What can be reason for fetcher thread for slow response.

Hi Erik & Prabhjot We are using Kafka-0.8.2.1 and old producer API with below config: request.required.acks=1 request.timeout.ms=2000 producer.type=sync On Kafka broker we are having: num.network.threads=8 num.io.threads=10 num.replica.fetchers=4 replica.fetch.max.bytes=2097154

Re: Auto preferred leader elections cause data loss?

Hi group, Sorry I'm confused with *min.insync.replicas: When a producer sets request.required.acks to -1, min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. *When talking about number of replicas, dose it include