Re: Partitions reassignment is failing in Kafka 1.1.0

Code snippet from ReassignPartitionsCommand.scala : "log_dirs" -> replicas.map(r => replicaLogDirAssignment.getOrElse(new TopicPartitionReplica(tp.topic, tp.partition, r), AnyLogDir)).asJava We know that the appearance of "any" was due to the OrElse clause. Arguably there is a bug in

How to fix ISR/URP when there is no new FetchRequest made?

We have URP reported for some empty topics, it was very annoying. The below screen shots shows both leader (1004) and follower (1001) has the exact same content. They probably get into this stage when some brokers were prematurely stopped and restarted. By looking at the code, looks like ISR

Kafka Streams - Shared Ktable State Store

Hi We are using kafka streams api. We have come across an issue which we need a short term fix for. We host 2 independent kafka clusters in 2 different data centres and these clusters are mirrored using mirrormaker. The problem this brings is that we cannot use kTables backed by a rocksDB

Re: Partitions reassignment is failing in Kafka 1.1.0

Thanks On Sat, Jun 30, 2018 at 12:36 PM, Manikumar wrote: > It will be taken as "any" directory for each replica, which means replica > will placed on any one of the > configured directory on that broker. > > Since it is "log_dirs" optional, you can remove from the json. > > On Sat, Jun 30,

Re: [kafka-clients] [VOTE] 2.0.0 RC1

Hi Manikumar, Thank you for pointing that out, I had forgotten to drop the old artifacts. New artifacts should be there now. Regards, Rajini On Sat, Jun 30, 2018 at 7:44 AM, Manikumar wrote: > looks like maven artifacts are not updated in the staging repo. They are > still at old timestamp.

Re: [External] Re: Is order of data is not maintained in Kafka, Is Kafka not suitable to do manage State / Transactional Scenarios such as Updating a Bank Account scenarios etc

Hi Malik, I think it depends on the strategy you want for your producers and consumers. Actually one topic one partition you'll have partition ordered guaranteed and consequently topic too. However, if you have several partitions for one topic then the events in your partitions will be ordered

Re: [VOTE] 1.0.2 RC1

+1 Ran test suite. On Fri, Jun 29, 2018 at 10:02 PM, Matthias J. Sax wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Kafka 1.0.2. > > This is a bug fix release addressing 27 tickets: >

Re: Visible intermediate state for KGroupedTable aggregation

On Sat, Jun 30, 2018 at 1:48 AM Matthias J. Sax wrote: > I see what you are saying -- this is what I meant by eventual > consistency guarantee. Understood. > What is unclear to me is, why you need to re-group the KTable? The issue > you describe only occurs for this operation. The transactions

Re: Partitions reassignment is failing in Kafka 1.1.0

It will be taken as "any" directory for each replica, which means replica will placed on any one of the configured directory on that broker. Since it is "log_dirs" optional, you can remove from the json. On Sat, Jun 30, 2018 at 12:02 PM Debraj Manna wrote: > It is problem on my side. The code

Re: [kafka-clients] [VOTE] 2.0.0 RC1

looks like maven artifacts are not updated in the staging repo. They are still at old timestamp. https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.11/2.0.0/ On Sat, Jun 30, 2018 at 12:06 AM Rajini Sivaram wrote: > Hello Kafka users, developers and client-developers,

Re: Partitions reassignment is failing in Kafka 1.1.0

It is problem on my side. The code was changing the replicas count but not the log_dirs. Since I am migrating from 0.10 this part of the code was not changed. I have a follow up question what is the default value of log_dirs if I don't specify it in reassignment.json ? On Sat, Jun 30, 2018 at