Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #91

See Changes: [github] MINOR: standardize rebalance related logging for easy discovery & debugging (#9295) -- [...truncated 3.35 MB...]

Jenkins build is back to normal : Kafka » kafka-trunk-jdk15 #90

See

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #90

See Changes: [github] KAFKA-10077: Filter downstream of state-store results in spurious tombstones (#9156) -- [...truncated 6.70 MB...]

Re: [VOTE] KIP-584: Versioning scheme for features

Hi Colin, Thanks for the feedback. Those are very good points. I have made the following changes to the KIP as you had suggested: 1. Included the `timeoutMs` field in the `UpdateFeaturesRequest` schema. The initial implementation won't be making use of the field, but we can always use it in the

Re: [VOTE] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hey All, So the main thing the EnvelopeRequest gives us is a way to avoid converting older API versions in order to attach the initial principal name and the clientId. It also saves the need to add the initial principal and client id as a tagged field to all of the forwarded protocols, which is

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

Hi, Colin, Thanks for the reply. 62. Thinking about this more, I am wondering what's our overall strategy for configs shared between the broker and the controller. For example, both the broker and the controller have to define listeners. So both need configs like listeners/advertised.listeners.

Re: [VOTE] KIP-584: Versioning scheme for features

Hi Jun, Thanks for the feedback. It's a very good point. I have now modified the KIP-584 write-up "goals" section a bit. It now mentions one of the goals as enabling rolling upgrades using a single restart (instead of 2). Also I have removed the text explicitly aiming for deprecation of IBP. Note

Re: [DISCUSS] KIP-630: Kafka Raft Snapshot

Hi, Jose, Thanks for the reply. A few more comments below. 20. Good point on metadata cache. I think we need to make a decision consistently. For example, if we decide that dedicated voter nodes don't serve metadata requests, then we don't need to expose the voters host/port to the client. Which

Re: [DISCUSS] KIP-671: Shutdown Streams Application when appropriate exception is thrown

I am wondering about the usage pattern of this new method. As already discussed, the method only works if there is at least one running thread... Do we have any sense how many apps actually run multi-threaded vs single-threaded? It seems that the feature might be quite limited without having a

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #89

See Changes: [github] KAFKA-10519; Add missing unit test for `VotedState` (#9337) [github] MINOR: Use the automated protocol for the Consumer Protocol's subscriptions and assignments (#8897)

Jenkins build is back to normal : Kafka » kafka-trunk-jdk8 #88

See

Re: [VOTE] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

On Fri, Sep 25, 2020, at 10:49, Boyang Chen wrote: > Hey Jun, > > On Fri, Sep 25, 2020 at 10:19 AM Jun Rao wrote: > > > Hi, Boyang, > > > > Does EnvelopeRequest avoid the need for IBP? How do we know if the > > controller supports EnvelopeRequest or not? > > > > Unfortunately, the

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

On Fri, Sep 25, 2020, at 10:17, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. > > 60. Yes, I think you are right. We probably need the controller id when a > broker starts up. A broker only stores the Raft leader id in the metadata > file. To do the initial fetch to the Raft leader, it

Re: [VOTE] KIP-584: Versioning scheme for features

On Tue, Sep 22, 2020, at 00:43, Kowshik Prakasam wrote: > Hi all, > > I wanted to let you know that I have made the following changes to the > KIP-584 write up. The purpose is to ensure the design is correct for a few > things which came up during implementation: > Hi Kowshik, Thanks for the

Re: [DISCUSS] KIP-630: Kafka Raft Snapshot

Thanks for the detailed feedback Jun. The changes are here: https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=158864763=25=24 Here is a summary of the change to the KIP: 1. Use end offset for snapshot and snapshot id. 2. Include default for all of the new configuration

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #89

See Changes: [github] KAFKA-10519; Add missing unit test for `VotedState` (#9337) [github] MINOR: Use the automated protocol for the Consumer Protocol's subscriptions and assignments (#8897)

Re: [VOTE] KIP-584: Versioning scheme for features

Hi, Kowshik, Thanks for the update. Those changes seem fine to me. One of the goals listed in this KIP is the removal of IBP in the future. Recent discussion in KIP-590 intends to expand the scope of IBP for forwarding requests to the controller. In light of this, is the goal of removing IBP

Re: [DISCUSS] KIP-673: Emit JSONs with new auto-generated schema

Hi Tom, Thanks for your input! 1. I'll add more details for the RequestConvertToJson and XYZJsonDataConverter classes. Hopefully it will be more clear, but just to answer your question, RequestConvertToJson does not return a XYZJsonDataConverter, but rather it returns a JsonNode which will be

Re: [VOTE] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hey Jun, On Fri, Sep 25, 2020 at 10:19 AM Jun Rao wrote: > Hi, Boyang, > > Does EnvelopeRequest avoid the need for IBP? How do we know if the > controller supports EnvelopeRequest or not? > > Unfortunately, the EnvelopeRequest is solving the inter-broker communication problem only. Admin

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #87

See Changes: [github] KAFKA-10516; Disable automatic retry of `THROTTLING_QUOTA_EXCEEDED` errors in the `kafka-topics` command (KIP-599) (#9334) --

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #88

See Changes: [github] KAFKA-10516; Disable automatic retry of `THROTTLING_QUOTA_EXCEEDED` errors in the `kafka-topics` command (KIP-599) (#9334) --

Re: [VOTE] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hi, Boyang, Does EnvelopeRequest avoid the need for IBP? How do we know if the controller supports EnvelopeRequest or not? Thanks, Jun On Thu, Sep 24, 2020 at 6:22 PM Boyang Chen wrote: > Hey Jason and Jun, > > thanks for the reply. Actually after some offline discussion, we have seen >

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

Hi, Colin, Thanks for the reply. 60. Yes, I think you are right. We probably need the controller id when a broker starts up. A broker only stores the Raft leader id in the metadata file. To do the initial fetch to the Raft leader, it needs to know the host/port associated with the leader id.

[jira] [Resolved] (KAFKA-10519) Unit tests for VotedState

[ https://issues.apache.org/jira/browse/KAFKA-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-10519. - Resolution: Fixed > Unit tests for VotedState > - > >

[jira] [Created] (KAFKA-10528) Make output of AK various tools consistent in case of errors

David Jacot created KAFKA-10528: --- Summary: Make output of AK various tools consistent in case of errors Key: KAFKA-10528 URL: https://issues.apache.org/jira/browse/KAFKA-10528 Project: Kafka

Re: [DISCUSS] KIP-516: Topic Identifiers

Hi Ismael, If you do not store it in a metadata file or in the directory structure would you then require the LeaderAndIsrRequest following startup to give you some notion of topic name in memory? We would still need this information for the older protocols, but perhaps this is what's meant by

[jira] [Resolved] (KAFKA-10516) Implement Topic Command changes

[ https://issues.apache.org/jira/browse/KAFKA-10516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-10516. - Resolution: Fixed > Implement Topic Command changes > --- > >

Re: [DISCUSS] KIP-516: Topic Identifiers

Hi Lucas, Why would you include the name and id? I think you'd want to remove the name from the directory name right? Jason's suggestion was that if you remove the name from the directory, then why would you need the id name mapping file? Ismael On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet

Re: [DISCUSS] KIP-673: Emit JSONs with new auto-generated schema

Hi Anastasia, Thanks for the KIP, I can certainly see the benefit of this. I have a few questions: 1. I think it would be helpful to readers to explicitly illustrate the RequestConvertToJson and XYZJsonDataConverter classes (e.g. with method signatures for one or two methods), because currently

Re: Question Regarding Offset Behavior When Calling Poll()

Hi, Sorry for the late reply, let me clarify on this. I am developing using Golang so I used a library based on librdkafka, and there's one function, ReadMesage(), which is a wrapper on top of the poll() function, except that it will only poll one message(record) at a time and return either one

Can I get a review for a documentation update (KAFKA-10473)?

Hi, Can I get a review from one of the commiters for this documentation update? I am adding docs for the following JMX metrics: kafka.log,type=Log,name=Size kafka.log,type=Log,name=NumLogSegments kafka.log,type=Log,name=LogStartOffset