Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2317

See

Re: [DISCUSS] Apache Kafka 3.5.2 release

Hi Matthias, I agree it's indeed a blocker for 3.5.2 to address CVE in RocksDB. Please let me know when the test is completed. Thank you. Luke On Sat, Oct 21, 2023 at 2:12 AM Matthias J. Sax wrote: > Thanks for the info Luke. > > We did backport all but one PR in the mean time. The missing PR

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.6 #96

See Changes: -- [...truncated 308761 lines...] > Task :connect:api:compileTestJava UP-TO-DATE > Task :connect:api:testClasses UP-TO-DATE > Task

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2316

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #84

See Changes: -- [...truncated 566093 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 183 > KTableSourceTopicRestartIntegrationTest >

[jira] [Resolved] (KAFKA-15378) Rolling upgrade system tests are failing

[ https://issues.apache.org/jira/browse/KAFKA-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15378. - Fix Version/s: 3.4.2 3.5.2 3.7.0

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2315

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.6 #95

See Changes: -- [...truncated 409138 lines...] Gradle Test Run :connect:runtime:test > Gradle Test Executor 43 >

[jira] [Resolved] (KAFKA-15651) Investigate auto commit guarantees during Consumer.assign()

[ https://issues.apache.org/jira/browse/KAFKA-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-15651. - Resolution: Not A Problem > Investigate auto commit guarantees during Consumer.assign() >

[jira] [Resolved] (KAFKA-15626) Replace verification guard object with an specific type

[ https://issues.apache.org/jira/browse/KAFKA-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15626. Resolution: Fixed > Replace verification guard object with an specific type >

[jira] [Created] (KAFKA-15663) Implement ClientTelemetryReporter which manages telemetry lifecyclye

Apoorv Mittal created KAFKA-15663: - Summary: Implement ClientTelemetryReporter which manages telemetry lifecyclye Key: KAFKA-15663 URL: https://issues.apache.org/jira/browse/KAFKA-15663 Project:

Re: [DISCUSS] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

Thank you Matthias, I will modify the KIP to eliminate this restriction. Sincerely, Hanyu On Fri, Oct 20, 2023 at 2:04 PM Hanyu (Peter) Zheng wrote: > Thank you Alieh, > > In these two new query types, I will remove 'get' from all getter method > names. > > Sincerely, > Hanyu > > On Fri, Oct

Re: [DISCUSS] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

Thank you Alieh, In these two new query types, I will remove 'get' from all getter method names. Sincerely, Hanyu On Fri, Oct 20, 2023 at 10:40 AM Matthias J. Sax wrote: > Thanks for the KIP Hanyu, > > One questions: > > > To address this inconsistency, we propose that KeyQuery should be >

[jira] [Created] (KAFKA-15662) Implement support for clientInstanceIds in Kafka Stream

Apoorv Mittal created KAFKA-15662: - Summary: Implement support for clientInstanceIds in Kafka Stream Key: KAFKA-15662 URL: https://issues.apache.org/jira/browse/KAFKA-15662 Project: Kafka

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #83

See Changes: -- [...truncated 563475 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 181 > TableTableJoinIntegrationTest > [caching enabled = true]

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2314

See Changes: -- [...truncated 217596 lines...] Gradle Test Run :streams:test > Gradle Test Executor 84 > DefaultStateUpdaterTest >

Re: [VOTE] KIP-988 Streams StandbyUpdateListener

+1 (non-binding) - great improvement, thanks Colt & Eduwer! On Tue, Oct 17, 2023 at 11:25 AM Guozhang Wang wrote: > +1 from me. > > On Mon, Oct 16, 2023 at 1:56 AM Lucas Brutschy > wrote: > > > > Hi, > > > > thanks again for the KIP! > > > > +1 (binding) > > > > Cheers, > > Lucas > > > > > > >

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #82

See Changes: -- [...truncated 562216 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 182 > TableTableJoinIntegrationTest > [caching enabled = true]

Re: [DISCUSS] Apache Kafka 3.5.2 release

Thanks for the info Luke. We did backport all but one PR in the mean time. The missing PR is a RocksDB version bump. We want to consider it for 3.5.2, because it addresses a CVE. Cf https://github.com/apache/kafka/pull/14216 However, RocksDB versions bumps are a little bit more tricky, and

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

Forgot one thing: We could also pass `currentLag()` into `onBachLoaded()` instead of end-offset. -Matthias On 10/20/23 10:56 AM, Matthias J. Sax wrote: Thanks for digging into this Bruno. The JavaDoc on the consumer does not say anything specific about `endOffset` guarantees: Get the

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

Thanks for digging into this Bruno. The JavaDoc on the consumer does not say anything specific about `endOffset` guarantees: Get the end offsets for the given partitions. In the default {@code read_uncommitted} isolation level, the end offset is the high watermark (that is, the offset of

Re: [DISCUSS] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

Thanks for the KIP Hanyu, One questions: To address this inconsistency, we propose that KeyQuery should be restricted to querying kv-stores only, ensuring that it always returns a plain V type, making the behavior of the aforementioned code more predictable. Similarly, RangeQuery should

Re: [DISCUSS] KIP-987: Connect Static Assignments

Hey Hector, That's a cool idea for the ConnectAssignor plugin. While this proposal could be viewed as an "assignor" problem that a custom assignor could solve, it's really about providing additional context to the assignor which isn't present currently. This lack of context would prevent a

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2313

See Changes: -- [...truncated 420448 lines...] Gradle Test Run :core:test > Gradle Test Executor 88 > ZkAclMigrationClientTest > testAclsMigrateAndDualWrite() STARTED

Re: [DISCUSS] KIP-987: Connect Static Assignments

Mickael, Thank you for discussing that rejected alternative, I was almost going to propose it. > I still find the proposed mechanism limited and I'm not sure it really > addressed the pain points I've experienced with Connect. I think that this KIP on its own is insufficient to solve the

RE: [DISCUSS] KIP-905: Broker interceptors

Hi David, I wonder if you considered interceptors that work not only on produce request, but on any request type the broker is receiving (apart from maybe internal request types). For example, being able to rewrite topic names in all the request types is essential to create a virtual clusters

RE: Re: [DISCUSS] KIP-905: Broker interceptors

Hi David and Ahmed, First, thank you David for the KIP. It would be very valuable for multiple use cases. Products like Conduktor Gateway [1] validate the demand and offer many potential use cases [2]. Now, I understand Ahmed's concerns about possible in-band interruptions, the are valid.

[VOTE] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

Hi Kafka Team. II'd like to call a vote on KIP-967: Support custom SSL configuration for Kafka Connect RestServer [1]. Discussion thread [2] was started more then 2 month ago and there was not any negative or critical comments. [1].

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.5 #81

See

[jira] [Created] (KAFKA-15661) KIP-951: Server side and protocol changes

Crispin Bernier created KAFKA-15661: --- Summary: KIP-951: Server side and protocol changes Key: KAFKA-15661 URL: https://issues.apache.org/jira/browse/KAFKA-15661 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

Hi Krishna, Overall I'm supportive of having an official docker image. I have a few questions: - Can you clarify the process of selecting the Java version? Is the proposal to only pick LTS versions? or to pick the highest version supported by Kafka? - Once a new Kafka version is released, what

[jira] [Resolved] (KAFKA-15428) Cluster-wide dynamic log adjustments for Connect

[ https://issues.apache.org/jira/browse/KAFKA-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-15428. --- Fix Version/s: 3.7.0 Resolution: Done > Cluster-wide dynamic log adjustments for

Re: [DISCUSS] KIP-978: Allow dynamic reloading of certificates with different DN / SANs

Please let me know if anyone has some more comments on this. If not, I will start the vote next week. Thanks & Regards Jakub On Wed, Sep 13, 2023 at 9:59 PM Jakub Scholz wrote: > Hi all, > > I would like to start the discussion about the KIP-978: Allow dynamic > reloading of certificates with

Re: [DISCUSS] KIP-987: Connect Static Assignments

Hi, I think different algorithms might work for different workload/scenarios. I have some thoughts that are somewhat tangential to this KIP: it might be a good idea to elevate the ConnectAssignor to the category of plugin, so users can provide their own implementation. The fact that there's

Re: Re: [DISCUSS] Apache Kafka 3.5.2 release

Hi Ryan, OK, I've backported it to 3.5 branch. I'll be included in v3.5.2. Thanks. Luke On Fri, Oct 20, 2023 at 7:43 AM Ryan Leslie (BLP/ NEW YORK (REMOT) < rles...@bloomberg.net> wrote: > Hi Luke, > > Hope you are well. Can you please include >

Re: [DISCUSS] Apache Kafka 3.5.2 release

Hi Matthias, I'm planning to have the 1st RC next week. Does that work for you? Should I defer one more week? Thanks. Luke On Wed, Oct 18, 2023 at 1:52 AM Matthias J. Sax wrote: > Thanks -- there is a few fixed for Kafka Streams we are considering to > cherry-pick to get into 3.5.2 release --

[jira] [Created] (KAFKA-15660) File-based Tiered Storage should delete folders upon topic deletion

Christo Lolov created KAFKA-15660: - Summary: File-based Tiered Storage should delete folders upon topic deletion Key: KAFKA-15660 URL: https://issues.apache.org/jira/browse/KAFKA-15660 Project: Kafka

Re: [VOTE] KIP-982: Enhance Custom KafkaPrincipalBuilder to Access SslPrincipalMapper and KerberosShortNamer

Hi, Thanks for the KIP. +1 (binding) Thanks, Manikumar On Fri, Oct 20, 2023 at 4:26 AM Raghu B wrote: > Hi everyone, > > I would like to start a vote on KIP-982, which proposed enhancements to > the Custom KafkaPrincipalBuilder to allow access to SslPrincipalMapper and > KerberosShortNamer.

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.5 #80

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2312

See Changes: -- [...truncated 327276 lines...] Gradle Test Run :core:test > Gradle Test Executor 88 > ZkMigrationClientTest > testEmptyWrite() STARTED Gradle Test Run

[jira] [Created] (KAFKA-15659) Flaky test RestoreIntegrationTest.shouldInvokeUserDefinedGlobalStateRestoreListener

Divij Vaidya created KAFKA-15659: Summary: Flaky test RestoreIntegrationTest.shouldInvokeUserDefinedGlobalStateRestoreListener Key: KAFKA-15659 URL: https://issues.apache.org/jira/browse/KAFKA-15659

Re: [DISCUSS] KIP-987: Connect Static Assignments

Hi Greg, Thanks for the reply. I still find the proposed mechanism limited and I'm not sure it really addressed the pain points I've experienced with Connect. As you said different tasks from a connector may have different workload. Connectors may also change the assignment of tasks at runtime

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

Hi, Matthias is correct that the end offsets are stored somewhere in the metadata of the consumer. More precisely, they are stored in the `TopicPartitionState`. However, I could not find public API on the consumer other than currentLag() that uses the stored end offsets. If I understand the

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

Hi Manikumar, Thanks for the feedback! 1. We propose the addition of a new directory named "docker" at the root of the repository, where all Docker-related code will be stored. A detailed directory structure has been added in the KIP. 2. We request the creation of an Apache Kafka repository

Re: [DISCUSS] KIP-974 Docker Image for GraalVM based Native Kafka Broker

Hi, > For the native AK docker image, we are considering '*kafka-local*' as it clearly signifies that this image is intended exclusively for local I am not sure, if there is any naming pattern for graalvm based images. Can we include "graalvm" to the image name like "kafka-graalvm-native". This

Re: [DISCUSS] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

Hey Hanyu, Thanks for the KIP. It seems good to me. Just one point: AFAIK, we are going to remove "get" from the name of all getter methods. Cheers, Alieh On Thu, Oct 19, 2023 at 5:44 PM Hanyu (Peter) Zheng wrote: > Hello everyone, > > I would like to start the discussion for KIP-992:

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

Hi Krishna, Vedarth, Thanks for the KIP. 1. Can we add directory structure of Docker Image related files in Kafka repo. 2. > Steps for the Docker image release will be included in the Release Process doc of Apache Kafka Can we list down the requirements (repos, accounts) for releasing images

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2311

See

Re: [VOTE] KIP-982: Enhance Custom KafkaPrincipalBuilder to Access SslPrincipalMapper and KerberosShortNamer

Hi Raghu, Thanks for the short KIP. +1(non-binding) On Thu, 19 Oct 2023 at 23:56, Raghu B wrote: > Hi everyone, > > I would like to start a vote on KIP-982, which proposed enhancements to > the Custom KafkaPrincipalBuilder to allow access to SslPrincipalMapper and > KerberosShortNamer. > >

[jira] [Created] (KAFKA-15658) Zookeeper 3.6.3 jar | CVE-2023-44981

masood created KAFKA-15658: -- Summary: Zookeeper 3.6.3 jar | CVE-2023-44981 Key: KAFKA-15658 URL: https://issues.apache.org/jira/browse/KAFKA-15658 Project: Kafka Issue Type: Bug