Re: Manual offset commit losing a message?

2016-05-06 Thread Hemanth Yamijala
Sorry for the distraction. I think I had a typo in the property name: it is auto.commit.enable and not auto.commit.enabled (no 'd' at the end). It seems to be working as expected with the correction. Thanks hemanth On Fri, May 6, 2016 at 6:26 PM, Hemanth Yamijala wrote: > Hi

Manual offset commit losing a message?

2016-05-06 Thread Hemanth Yamijala
Hi all, I am using Kafka 0.9.0.1 and the old high level consumer API. I wanted to implement atleast once semantics in my app, and looking around it seemed like the right way to do it is to: - set "auto.commit.enabled=false" for the consumer properties. - Then, issue a ConsumerConnector.commitOff

Re: Ephemeral ports for Kafka broker

2015-11-12 Thread Hemanth Yamijala
Thanks for a quick response. On 12-Nov-2015 16:28, "Gerard Klijs" wrote: > Hi Hemanth, it was introduced on 04/04/2015 at github, so after the 0.8.2.0 > version, it will be part of the 0.9.0.0 release. > > On Thu, Nov 12, 2015 at 8:39 AM Hemanth Yamijala > wrote: >

Re: Ephemeral ports for Kafka broker

2015-11-11 Thread Hemanth Yamijala
ava wrote: > Passing 0 as the port should let you do this. This is how we get the tests > to work without assuming a specific port is available. The > KafkaServer.boundPort(SecurityProtocol) method can be used to get the port > that was bound. > > -Ewen > > On Tue, Nov 10,

Ephemeral ports for Kafka broker

2015-11-10 Thread Hemanth Yamijala
Hello, Is there any way to specify that the Kafka broker should bind to any available port. This could be used, maybe for an embedded Kafka instance, etc. Thanks Hemanth

Re: Setting log.default.flush.interval.ms and log.default.flush.scheduler.interval.ms

2014-09-16 Thread Hemanth Yamijala
e lower in the case of the lower flush interval, but they seem to be going up steadily. Is this something to monitor more closely over a longer period of time ? Can you please help me to interpret these results ? Thanks hemanth On Mon, Sep 15, 2014 at 1:58 PM, Hemanth Yamijala wrote: > Th

Re: Setting log.default.flush.interval.ms and log.default.flush.scheduler.interval.ms

2014-09-15 Thread Hemanth Yamijala
eventually lose data. > > Thanks, > Neha > > On Thu, Sep 11, 2014 at 5:48 PM, Hemanth Yamijala > wrote: > > > Neha, > > > > Thanks. We are on 0.7.2. I have written on another thread on the list > here > > about one of the reasons we are stuck - the a

Re: Setting log.default.flush.interval.ms and log.default.flush.scheduler.interval.ms

2014-09-11 Thread Hemanth Yamijala
d mostly be fine. You might want to keep a close tab on how > your iostat numbers are doing, to be sure. > > On Wed, Sep 10, 2014 at 5:46 PM, Hemanth Yamijala > wrote: > > > Thanks Jun. > > > > On Thu, Sep 11, 2014 at 4:13 AM, Jun Rao wrote: > >

Re: Setting log.default.flush.interval.ms and log.default.flush.scheduler.interval.ms

2014-09-10 Thread Hemanth Yamijala
Thanks Jun. On Thu, Sep 11, 2014 at 4:13 AM, Jun Rao wrote: > As long as the I/O load is reasonable, this is probably ok. > > Thanks, > > Jun > > On Wed, Sep 10, 2014 at 4:59 AM, Hemanth Yamijala > wrote: > > > Hi folks, > > > > In order to

Setting log.default.flush.interval.ms and log.default.flush.scheduler.interval.ms

2014-09-10 Thread Hemanth Yamijala
Hi folks, In order to meet latency requirements for a system we are building, we tested with different values of the above two parameters and found that settings as low as 100 work best for us, balancing the required throughput and latencies. I just wanted to check if 100 is a sane value, notwith

Php client for Kafka 0.8

2014-09-09 Thread Hemanth Yamijala
Hi folks, One of things stopping us from upgrading version of Kafka from 0.7 to 0.8 is the absence of a native PHP client. A search points to this: https://github.com/salebab/phpkafka. Does anyone have experience with it ? We require a PHP client due to high throughput, low latency existing syste

Re: Intellij setup for Kafka 0.7.x

2014-09-02 Thread Hemanth Yamijala
onsult IDE log for more details (Help | Show Log) === There was a note on setting the Scala version to 2.9.2 which I did, but it didn't seem to improve. Thanks Hemanth On Tue, Sep 2, 2014 at 5:50 PM, Hemanth Yamijala wrote: > Hi, > > I am trying to setup Kafka 0.7.2 project in Intel

Intellij setup for Kafka 0.7.x

2014-09-02 Thread Hemanth Yamijala
Hi, I am trying to setup Kafka 0.7.2 project in Intellij Idea 13 CE. The wiki instructions for developers seems to be pointing to trunk. Since the place I'm working for is using the older version, I was thinking of setting that version up in Intellij. Are there instructions on how to do that ? T