Kafka offset

Hi   Does  fromOffset reset to zero when  auto.offset.reset is smallest? Or Is it depends  on auto commit  interval  when  not  explicitly  committing via application? Sent from Samsung Mobile.

replica-assignment tool not working when leader not available

Can’t reassign partition of leader (-1) is unavailable. Replication-factor: 1 Initial brokers: 2 [1001, 1002] unclean.leader.election.enable=true Say broker 1002 went down. As expected kafka-topics describe shows Topic: my_topic Partition: 5Leader: -1 Replicas: 1002

Networking errors and durability settings

Hi everyone, Yesterday we had lots of network failures running our Kafka cluster (0.9.0.1 ~40 nodes). We run everything using the higher durability settings in order to avoid in data loss, producers use all/-1 ack, topics/brokers have min insync replicas = 2. unclean leader election = false, and

Re: Build Issue with Gradle

Hi, Thanks a lot! am able to build and import the projects in Eclipse now. Thanks, On Tue, Aug 23, 2016 at 5:51 PM, Sankar Narayanan wrote: > Hi, > > I got your response as below. > > "Hi Sankar, > > It looks like Kafka's build scripts are not compatible with

unexpected consumer rebalance 0.9.0.1

Hi I am experiencing the following issue in kafka 0.9.0.1, I have a consumer, that is in a consumer group alone processing and commiting the offsets and at one point the group does a rebalance (I don't know why) and removes the group. The weird situation is that it seems that the consumer is

Re: Build Issue with Gradle

Hi Sankar, The following PR fixes the issue: https://github.com/apache/kafka/pull/1774 Ismael On Mon, Aug 22, 2016 at 10:00 AM, Sankar Narayanan < meetshankarh...@gmail.com> wrote: > Hi, > > I am facing the below issue while trying to build kafka using gradle. can > you please help? > >

Re: Prevent broker from leading topic partitions

Alexey, Thanks for the response. Doesn't leader rebalancing just allow the preferred leader to become leader again once it's recovered? Node 2 would still become leader for all partitions if node 1 failed. That's not exactly what I'm looking to achieve. I need to ensure that node 2 never

RE: Build Issue with Gradle

gradle hides alot of details such as how the package jars (how does assembly/package include external resources such as dlls?) also the gradle scala -> java conversion was empty so I retasked kafka core and client modules to regular maven pom.xml I am currently correcting my way thru 100+

Re: Build Issue with Gradle

Oh, that's interesting. It looks like the Gradle Wrapper's jar file was intentionally removed from the kafka source tree (https://issues.apache.org/jira/browse/KAFKA-2098), which would cause this error. The README file in the repo says to run "gradle" first, which will install the wrapper

Re: Build Issue with Gradle

Hi, I got your response as below. "Hi Sankar, It looks like Kafka's build scripts are not compatible with Gradle 3.0. I'd suggest using the gradle wrapper (./gradlew) included in the Kafka repo, which will automatically install Gradle 2.13 which is compatible with the build scripts. Mathieu",

Re: Offsets getting lost if no messages sent for a long time

I don't know the answer to the second question, if you don't use (much) auto-generated id's for the consumer group you should be ok, since it's a compacted topic after all, you might want to check if the compaction is on. We set the offsets.retention.minutes to a week without a problem. On Tue,

Re: Offsets getting lost if no messages sent for a long time

Might be easier to handle duplicate messages as opposed to handling long periods of time without messages. Michael > On 22 Aug 2016, at 15:55, Misra, Rahul wrote: > > Hi, > > Can anybody provide any guidance on the following: > > 1. Given a limited set of groups

Re: Embedded Kafka Broker Health Check

Hi, Can someone help on this topic? Thank you Enrico Il Mer 27 Lug 2016 12:26 Enrico Olivelli - Diennea < enrico.olive...@diennea.com> ha scritto: > Hi, > I'm running Kafka launching KafkaServerStartable inside my JVM (I'm using > version 0.10.0.0). > > I'm accessing the internal KafkaServer