Re: [VOTE] KIP-228 Negative record timestamp support

Sorry for getting in the game this late, and on the wrong thread! I think negative timestamps makes sense and is a good addition, but I have a couple of concerns with the proposal: 1. I believe any change to the protocol format or semantics require a protocol bump, in this case for

Re: Vote for KIP-393 (Fix time windowed serde to deserialize changelog topic)

+1 (non-binding) On Wed, Dec 5, 2018 at 4:51 PM Guozhang Wang wrote: > Hello Shawn, > > Thanks for the writeup. I've made a pass over it and here are some minor > comments: > > 1) As we discussed in the PR: https://github.com/apache/kafka/pull/5307, > the public APIs that we will add is > > In

[jira] [Created] (KAFKA-7711) Add a bounded flush() API to Kafka Producer

kun du created KAFKA-7711: - Summary: Add a bounded flush() API to Kafka Producer Key: KAFKA-7711 URL: https://issues.apache.org/jira/browse/KAFKA-7711 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-398: Support reading trust store from classpath

Hi Noa, Thanks for the KIP. A few comments/questions: 1. If we support filenames starting with `classpath:` by requiring `file:`prefix, then we are presumably not supporting files starting `file:`. Not necessarily an issue, but we do need to document any restrictions. 2. On the broker-side,

Re: [VOTE] KIP-354 Time-based log compaction policy

Thanks Colin, Dong, and Joel for voting. I have masked this KIP as accepted. The pull request is up: https://github.com/apache/kafka/pull/6009 Xiongqi (Wesley) Wu On Thu, Dec 6, 2018 at 10:47 AM Joel Koshy wrote: > +1 on the updated KIP. > > On Wed, Dec 5, 2018 at 11:56 AM Dong Lin wrote:

Kafka replication error

Team, We are observing ISR shrink and expand very frequently. In the logs of the follower, below errors are observed: [2018-12-06 20:00:42,709] WARN [ReplicaFetcherThread-2-15], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@a0f9ba9 (kafka.server.ReplicaFetcherThread)

Re: Kafka replication error

+users On Thu, Dec 6, 2018 at 9:01 PM Suman B N wrote: > Team, > > We are observing ISR shrink and expand very frequently. In the logs of the > follower, below errors are observed: > > [2018-12-06 20:00:42,709] WARN [ReplicaFetcherThread-2-15], Error in fetch >

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

Nice proposal. Some comments. On the section around cycle detection. I would like to see support for this to be done by hops, as well e.g. using approach is to use a header for the number of hops, as the mm2 replicates it increases the hop count and you can make the mm2 configurable to only

[jira] [Resolved] (KAFKA-7709) ConcurrentModificationException occurs when iterating through multiple partitions in Sender.getExpiredInflightBatches

[ https://issues.apache.org/jira/browse/KAFKA-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7709. -- Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 Thanks [~markcho] for

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

Re hops to stop the cycle and to allow a range of multi cluster topologies, see https://www.rabbitmq.com/federated-exchanges.html where very similar was done in rabbit. On 12/7/18, 12:47 AM, "Michael Pearce" wrote: Nice proposal. Some comments. On the section around cycle

[jira] [Resolved] (KAFKA-7704) kafka.server.ReplicaFetechManager.MaxLag.Replica metric is reported incorrectly

[ https://issues.apache.org/jira/browse/KAFKA-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-7704. Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 Merged to trunk and 2.1. >

Build failed in Jenkins: kafka-trunk-jdk11 #144

See Changes: [wangguoz] KAFKA-7709: Fix ConcurrentModificationException when retrieving expired -- [...truncated 2.24 MB...]

Kafka Cluster Setup

Hi, I have a use case I want to set up a Kafka cluster initially at the starting I have 1 Kafka Broker(A) and 1 Zookeeper Node. So below mentioned are my queries: - On adding a new Kafka Broker(B) to the cluster. Will all data present on broker A will be distributed automatically? If not

Re: Kafka Cluster Setup

Hi Abhimanyu, I have answered your questions inline, but before that I just want to emphasize the notion of topics and partitions that are critical to Kafka's resiliency and scalability. Topics in Kafka can have multiple partitions. Each partition can be stored on one broker only. But the number

Re: Kafka Cluster Setup

Hi Vahid, I missed one more point. - Can I have two brokers at the same time running at multiple versions just for Kafka version upgrade? Regards, Abhimanyu On Fri, Dec 7, 2018 at 1:03 PM Abhimanyu Nagrath wrote: > Thanks for clarifying all the points . Just a small doubt regarding my

Re: Kafka 1.1.0 no longer available for download?

You can download from here https://kafka.apache.org/downloads/ https://archive.apache.org/dist/kafka/1.1.0/kafka_2.12-1.1.0.tgz On Wed, Dec 5, 2018, at 2:12 PM, David Glasser wrote: > It looks like 1.1.0 is no longer available at > https://www.apache.org/dist/kafka/ > > Is this intentional?

Re: Kafka Cluster Setup

Thanks for clarifying all the points . Just a small doubt regarding my second query: - *Not let's suppose somehow the case! is solved my data is distributed on both the brokers. Now due to some maintenance issue, I want to take down the server B.* - *How to transfer the data of

Build failed in Jenkins: kafka-trunk-jdk8 #3247

See Changes: [wangguoz] KAFKA-7709: Fix ConcurrentModificationException when retrieving expired -- [...truncated 4.49 MB...]

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

Ah, good point. Should we consider altering the serializer interface to permit not sending the record? On Wed, Dec 5, 2018 at 9:23 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Matt, > > That's a good point. If these cases are handled in the serializer, then > one

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

Matt, I agree with Matthias on not to altering the serializer as it's used by multiple components. Matthias, - the proposed method accepts a `ProducerRecord` -- it might be good to explain why this cannot be done in a type safe way (ie, missing generics) To accept different types of

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

Well, that's exactly the point. The serializer should not be altered IMHO because this would have impact on other components. Also, for applications that use KafkaProducer directly, they can catch any serialization exception and react to it. Hence, I don't don't see a reason to change the

Jenkins build is back to normal : kafka-2.1-jdk8 #76

See

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

>> To accept different types of records from multiple topologies, I have to >> define the ProducerRecord without generics. Yes. It does make sense. My point was, that the KIP should mention/explain this explicitly to allow other not familiar with the code base to understand it more easily :)

Re: [VOTE] KIP-354 Time-based log compaction policy

+1 on the updated KIP. On Wed, Dec 5, 2018 at 11:56 AM Dong Lin wrote: > Thanks for the update. +1 (binding) > > On Wed, Dec 5, 2018 at 8:19 AM Colin McCabe wrote: > > > Thanks, Xiongqi Wu. +1 (binding) > > > > regards, > > Colin > > > > > > On Tue, Dec 4, 2018, at 20:58, xiongqi (wesley) wu

Re: Kafka replication error

Guys, Another observation is 90% of under-replicated partitions have the same node as the follower. *Any help in here is very much appreciated. We have very less time to stabilize kafka. Thanks a lot in advance.* -Suman On Thu, Dec 6, 2018 at 9:08 PM Suman B N wrote: > +users > > On Thu, Dec