[jira] [Created] (KAFKA-15594) Add 3.6.0 to streams upgrade/compatibility tests

Satish Duggana created KAFKA-15594: -- Summary: Add 3.6.0 to streams upgrade/compatibility tests Key: KAFKA-15594 URL: https://issues.apache.org/jira/browse/KAFKA-15594 Project: Kafka Issue

[jira] [Created] (KAFKA-15593) Add 3.6.0 to broker/client upgrade/compatibility tests

Satish Duggana created KAFKA-15593: -- Summary: Add 3.6.0 to broker/client upgrade/compatibility tests Key: KAFKA-15593 URL: https://issues.apache.org/jira/browse/KAFKA-15593 Project: Kafka

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

Quick addendum. Some DSL operator use `range` and seems to rely on ascending order, too... Of course, if we say `range()` has no ordering guarantee, we would add `forwardRange()` and let the DSL use `forwardRange`, too. The discussion of course also applies to `all()` and `reveserAll()`, and

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

Thanks for raising this question Hanyu. Great find! My interpretation is as follows (it's actually a warning signal that the API contract is not better defined, and we should fix this by extending JavaDocs and docs on the web page about it). We have existing `range()` and `reverseRange()`

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

Thanks for the update. To retrieve the latest value(s), the user must call just the asOf method with the MAX value (asOf(MAX)). The same applies to KIP-968. Do you think it is clumsy, Matthias? Well, in KIP-968 calling `asOf` and passing in a timestamp is optional, and default

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

Sohpie— Thank you very much for such a detailed review of the KIP. It might actually be longer than the original KIP in the first place! 1. Ack'ed and fixed. 2. Correct, this is a confusing passage and requires context: One thing on our list of TODO's regarding reliability is to determine how

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2276

See

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

Thanks for the update! Some thoughts about changes you made and open questions you raised: 10) About asOf vs until: I was just looking into `WindowKeyQuery`, `WindowRangeQuery` and also `ReadOnlyWindowStore` interfaces. For those, we use "timeFrom" and "timeTo", so it seems best to

[jira] [Created] (KAFKA-15592) Member does not need to always try to join a group when a groupId is configured

Philip Nee created KAFKA-15592: -- Summary: Member does not need to always try to join a group when a groupId is configured Key: KAFKA-15592 URL: https://issues.apache.org/jira/browse/KAFKA-15592 Project:

Security for Kafka

Hello. I am trying to implement Quorum TLS by following the instructions in https://zookeeper.apache.org/doc/r3.5.7/zookeeperAdmin.html#Quorum+TLS, but I keep on encountering the following errors after doing the second rolling restart where sslQuorum set to true. - [2023-10-11 05:46:03,250]

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.6 #90

See Changes: -- [...truncated 307140 lines...] > Task :connect:api:testClasses UP-TO-DATE > Task :connect:json:generateMetadataFileForMavenJavaPublication > Task

Re: [DISCUSS] KIP-714: Client metrics and observability

In can answer 130 and 131. 130) We cannot guarantee that all clients are already initialized due to race conditions. We plan to not allow calling `KafkaStreams#clientsInstanceIds()` when the state is not RUNNING (or REBALANCING) though -- guess this slipped on the KIP and should be added?

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2275

See Changes: -- [...truncated 315649 lines...] Gradle Test Run :streams:test > Gradle Test Executor 84 > TasksTest >

[jira] [Created] (KAFKA-15591) Trogdor produce workload reports errors in KRaft mode

Xi Yang created KAFKA-15591: --- Summary: Trogdor produce workload reports errors in KRaft mode Key: KAFKA-15591 URL: https://issues.apache.org/jira/browse/KAFKA-15591 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-714: Client metrics and observability

Hi, Andrew, Thanks for the updated KIP. Just a few more minor comments. 130. KafkaStreams.clientsInstanceId(Duration timeout): Does it wait for all consumer/producer/adminClient instances to be initialized? Are all those instances created during KafkaStreams initialization? 131. Why does

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi David, Thanks for the comment. Yes, we can separate the ELR enablement from the metadata version. It is also helpful to avoid blocking the following MV releases if the user is not ready for ELR. One thing to correct is that, the Unclean recovery is controlled by

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

One thing we should consider is a static config to totally enable/disable the ELR feature. If I understand the KIP correctly, we can effectively disable the unclean recovery by setting the recovery strategy config to "none". This would make development and rollout of this feature a bit smoother.

[jira] [Resolved] (KAFKA-15571) StateRestoreListener#onRestoreSuspended is never called because wrapper DelegatingStateRestoreListener doesn't implement onRestoreSuspended

[ https://issues.apache.org/jira/browse/KAFKA-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-15571. Resolution: Fixed > StateRestoreListener#onRestoreSuspended is never

Re: [VOTE] KIP-714: Client metrics and observability

This looks great! +1 (binding) Sophie On Wed, Oct 11, 2023 at 1:46 PM Matthias J. Sax wrote: > +1 (binding) > > On 9/13/23 5:48 PM, Jason Gustafson wrote: > > Hey Andrew, > > > > +1 on the KIP. For many users of Kafka, it may not be fully understood > how > > much of a challenge client

[jira] [Resolved] (KAFKA-15589) Flaky kafka.server.FetchRequestTest

[ https://issues.apache.org/jira/browse/KAFKA-15589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15589. Resolution: Duplicate Duplicate of https://issues.apache.org/jira/browse/KAFKA-15566 >

[jira] [Created] (KAFKA-15590) Replica.updateFetchState should also fence updates with stale leader epoch

Calvin Liu created KAFKA-15590: -- Summary: Replica.updateFetchState should also fence updates with stale leader epoch Key: KAFKA-15590 URL: https://issues.apache.org/jira/browse/KAFKA-15590 Project:

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2274

See Changes: -- [...truncated 316383 lines...] Gradle Test Run :streams:test > Gradle Test Executor 77 > TasksTest > shouldVerifyIfPendingTaskToRecycleExist() STARTED

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.6 #89

See Changes: -- [...truncated 204709 lines...] Gradle Test Run :streams:test > Gradle Test Executor 84 > TasksTest >

Re: [VOTE] KIP-714: Client metrics and observability

+1 (binding) On 9/13/23 5:48 PM, Jason Gustafson wrote: Hey Andrew, +1 on the KIP. For many users of Kafka, it may not be fully understood how much of a challenge client monitoring is. With tens of clients in a cluster, it is already difficult to coordinate metrics collection. When there are

Re: [DISCUSS] KIP-714: Client metrics and observability

Thanks! On 10/10/23 11:31 PM, Andrew Schofield wrote: Matthias, Yes, I think that’s a sensible way forward and the interface you propose looks good. I’ll update the KIP accordingly. Thanks, Andrew On 10 Oct 2023, at 23:01, Matthias J. Sax wrote: Andrew, yes I would like to get this

[jira] [Created] (KAFKA-15589) Flaky kafka.server.FetchRequestTest

Justine Olshan created KAFKA-15589: -- Summary: Flaky kafka.server.FetchRequestTest Key: KAFKA-15589 URL: https://issues.apache.org/jira/browse/KAFKA-15589 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-15588) Purge the unsent offset commits/fetches when the member is fenced

Philip Nee created KAFKA-15588: -- Summary: Purge the unsent offset commits/fetches when the member is fenced Key: KAFKA-15588 URL: https://issues.apache.org/jira/browse/KAFKA-15588 Project: Kafka

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.6 #88

See Changes: -- [...truncated 307216 lines...] streams-9: SMOKE-TEST-CLIENT-CLOSED streams-7: SMOKE-TEST-CLIENT-CLOSED streams-2: SMOKE-TEST-CLIENT-CLOSED streams-4:

[jira] [Created] (KAFKA-15587) Limit the number of records per batch in GroupCoordinatorShard#cleanupGroupMetadata

Jeff Kim created KAFKA-15587: Summary: Limit the number of records per batch in GroupCoordinatorShard#cleanupGroupMetadata Key: KAFKA-15587 URL: https://issues.apache.org/jira/browse/KAFKA-15587 Project:

[jira] [Created] (KAFKA-15586) Clean shutdown detection, server side

Calvin Liu created KAFKA-15586: -- Summary: Clean shutdown detection, server side Key: KAFKA-15586 URL: https://issues.apache.org/jira/browse/KAFKA-15586 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-15585) DescribeTopic API

Calvin Liu created KAFKA-15585: -- Summary: DescribeTopic API Key: KAFKA-15585 URL: https://issues.apache.org/jira/browse/KAFKA-15585 Project: Kafka Issue Type: Sub-task Reporter:

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

Hi all, Thanks for the votes! I'll cast a final +1 myself and close the vote out. This KIP passes with the following +1 votes (and no +0 or -1 votes): • Greg Harris (binding) • Yash Mayya (binding) • Federico Valeri • Mickael Maison (binding) • hudeqi • Hector Geraldino • Chris Egerton

[jira] [Created] (KAFKA-15584) ELR leader election

Calvin Liu created KAFKA-15584: -- Summary: ELR leader election Key: KAFKA-15584 URL: https://issues.apache.org/jira/browse/KAFKA-15584 Project: Kafka Issue Type: Sub-task Reporter:

[jira] [Created] (KAFKA-15583) High watermark can only advance if ISR size is larger than min ISR

Calvin Liu created KAFKA-15583: -- Summary: High watermark can only advance if ISR size is larger than min ISR Key: KAFKA-15583 URL: https://issues.apache.org/jira/browse/KAFKA-15583 Project: Kafka

[jira] [Created] (KAFKA-15582) Clean shutdown detection, broker side

Calvin Liu created KAFKA-15582: -- Summary: Clean shutdown detection, broker side Key: KAFKA-15582 URL: https://issues.apache.org/jira/browse/KAFKA-15582 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-15581) Introduce ELR

Calvin Liu created KAFKA-15581: -- Summary: Introduce ELR Key: KAFKA-15581 URL: https://issues.apache.org/jira/browse/KAFKA-15581 Project: Kafka Issue Type: Sub-task Reporter: Calvin

[jira] [Created] (KAFKA-15580) KIP-966: Unclean Recovery

Calvin Liu created KAFKA-15580: -- Summary: KIP-966: Unclean Recovery Key: KAFKA-15580 URL: https://issues.apache.org/jira/browse/KAFKA-15580 Project: Kafka Issue Type: New Feature

[jira] [Created] (KAFKA-15579) KIP-966: Eligible Leader Replicas

Calvin Liu created KAFKA-15579: -- Summary: KIP-966: Eligible Leader Replicas Key: KAFKA-15579 URL: https://issues.apache.org/jira/browse/KAFKA-15579 Project: Kafka Issue Type: New Feature

Re: [VOTE] KIP-960: Support single-key_single-timestamp interactive queries (IQv2) for versioned state stores

+1 (binding) Thanks for the kip Alieh! Walker On Wed, Oct 11, 2023 at 3:52 AM Bruno Cadonna wrote: > Thanks for the KIP, Alieh! > > +1 (binding) > > Best, > Bruno > > On 10/10/23 1:14 AM, Matthias J. Sax wrote: > > One more nit: as discussed on the related KIP-698 thread, we should not > >

[jira] [Created] (KAFKA-15578) Run System Tests for Old protocol in the New Coordinator

Ritika Muduganti created KAFKA-15578: Summary: Run System Tests for Old protocol in the New Coordinator Key: KAFKA-15578 URL: https://issues.apache.org/jira/browse/KAFKA-15578 Project: Kafka

Re: [VOTE]KIP-966: Eligible Leader Replicas

Hi, The KIP has received 3 binding votes from Justine Olshan, Jun Rao and Colin McCabe. Also thanks to Jeff Kim, Jack Vanlightly, Artem Livshits, David Arthur, David Jacot for the comments to make the KIP better. Thanks for the help! On Mon, Oct 9, 2023 at 10:20 AM Calvin Liu wrote: > Hi Colin,

[jira] [Resolved] (KAFKA-15221) Potential race condition between requests from rebooted followers

[ https://issues.apache.org/jira/browse/KAFKA-15221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-15221. - Fix Version/s: (was: 3.5.2) Resolution: Fixed > Potential race condition

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2273

See

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi Jun -Good catch, yes, we don't need the -1 in the DescribeTopicRequest. -No new value is added. The LeaderRecoveryState will still be set to 1 if we have an unclean leader election. The unclean leader election includes the old random way and the unclean recovery. During the unclean recovery,

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

Hey folks! We've incorporated all the points discussed into the KIP. Additionally, we've included a section detailing the Release Process of the Docker image. We value your feedback, so please take some time to review it. Once we have your inputs, we can proceed with the voting process. Thanks

Re: [DISCUSS] Road to Kafka 4.0

I think JBOD definitely needs to be before 4.0. That has been a blocker issue this entire time for me and my team and I'm sure others. While Kraft has been technically "production ready" for a while, I haven't been able to upgrade because of missing JBOD support. On Wed, Oct 11, 2023 at 12:15 PM

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

Hey Colt! Thanks for the KIP -- this will be a great addition to Streams, I can't believe we've gone so long without this. Overall the proposal makes sense, but I had a handful of fairly minor questions and suggestions/requests 1. Seems like the last sentence in the 2nd paragraph of the

Re: [DISCUSS] Road to Kafka 4.0

Hi Luke, This is a good discussion. And there is a lot more to it than KRaft. With regards to KRaft, there are two separate items: 1. Bugs 2. Missing features when compared to ZK When it comes to bugs, I don't see why 4.0 is particularly relevant. KRaft has been considered production-ready for

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi, Calvin, Another thing. Currently, when there is an unclean leader election, we set the LeaderRecoveryState in PartitionRecord and PartitionChangeRecord to 1. With the KIP, will there be new values for LeaderRecoveryState? If not, when will LeaderRecoveryState be set to 1? Thanks, Jun On

Re: [ANNOUNCE] Apache Kafka 3.6.0

Thanks Satish for all your hard work as release manager, and thanks to everyone else for their contributions! On Wed, Oct 11, 2023 at 6:07 AM Mickael Maison wrote: > Thanks Satish and to everyone who contributed to this release! > > Mickael > > > On Wed, Oct 11, 2023 at 11:09 AM Divij Vaidya >

Re: [DISCUSS] KIP-932: Queues for Kafka

Hi Jack, Thanks for your comments. I have added a new section on Log Retention which describes the behaviour of the SPSO as the LSO advances. That makes total sense and was an omission from the KIP. I have added the other ideas as potential future work. I do like the idea of having the SPSO

Re: Re: Re: [DISCUSS] KIP-972: Add the metric of the current running version of kafka

Hello, do we have a metric showing the uptime? We could tag that metric with version information as well. I like the idea of adding the version as a tag as well. However, I am not inclined to tag each metric with a KafkaVersion information. We could discuss which metrics could be tagged but let's

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2272

See Changes: -- [...truncated 420380 lines...] Gradle Test Run :streams:test > Gradle Test Executor 84 > TasksTest >

[DISCUSS] Road to Kafka 4.0

Hi all, While Kafka 3.6.0 is released, I’d like to start the discussion for the “road to Kafka 4.0”. Based on the plan in KIP-833 , the next release 3.7 will be

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

Thanks for the updates, Alieh! The example in the KIP uses the allVersions() method which we agreed to remove. Regarding your questions: 1. asOf vs. until: I am fine with both but slightly prefer until. 2. What about KeyMultiVersionsQuery, KeyVersionsQuery (KIP-960 would then be

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2271

See

Re: [REVIEW REQUEST] ReassignPartitionCommand scala -> java

Hello, Team. Just a friendly reminder. Huge scope of work are waiting to be finished with this PR. Please, join the review. > 5 окт. 2023 г., в 16:49, Николай Ижиков написал(а): > > Hello. > > PR got some review and approves from non committers. > CI is OK. > Dear committers, please, join

Re: [ANNOUNCE] Apache Kafka 3.6.0

Thanks Satish and to everyone who contributed to this release! Mickael On Wed, Oct 11, 2023 at 11:09 AM Divij Vaidya wrote: > > Thank you for all the hard work, Satish. > > Many years after the KIP-405 was written, we have it implemented and > finally available for beta testing for the users.

[jira] [Resolved] (KAFKA-15577) Reload4j | CVE-2022-45868

[ https://issues.apache.org/jira/browse/KAFKA-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-15577. --- Resolution: Not A Problem > Reload4j | CVE-2022-45868 > - > >

Re: [VOTE] KIP-960: Support single-key_single-timestamp interactive queries (IQv2) for versioned state stores

Thanks for the KIP, Alieh! +1 (binding) Best, Bruno On 10/10/23 1:14 AM, Matthias J. Sax wrote: One more nit: as discussed on the related KIP-698 thread, we should not use `get` as prefix for the getters. So it should be `K key()` and `Optional asOfTimestamp()`. Otherwise the KIP LGTM.

[jira] [Created] (KAFKA-15577) Reload4j | CVE-2022-45868

masood created KAFKA-15577: -- Summary: Reload4j | CVE-2022-45868 Key: KAFKA-15577 URL: https://issues.apache.org/jira/browse/KAFKA-15577 Project: Kafka Issue Type: Bug Reporter: masood

Re: Re: Re: [DISCUSS] KIP-972: Add the metric of the current running version of kafka

Just to chime in here since I recently went through a similar thing, I support adding the version as a tag instead of introducing an entirely new metric for this. In fact I just implemented exactly this in a project that uses Kafka, for these reasons: 1. Adding the version as a tag means that all

Re: [DISCUSS] KIP-714: Client metrics and observability

Matthias, Yes, I think that’s a sensible way forward and the interface you propose looks good. I’ll update the KIP accordingly. Thanks, Andrew > On 10 Oct 2023, at 23:01, Matthias J. Sax wrote: > > Andrew, > > yes I would like to get this change into KIP-714 right way. Seems to be >

Re: [ANNOUNCE] Apache Kafka 3.6.0

Thanks for running the release, Satish! BTW, 3.6.0 should be a major release, not a minor one. :) Luke On Wed, Oct 11, 2023 at 1:39 PM Satish Duggana wrote: > The Apache Kafka community is pleased to announce the release for > Apache Kafka 3.6.0 > > This is a minor release and it includes

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2270

See Changes: -- [...truncated 315908 lines...] Gradle Test Run :streams:test > Gradle Test Executor 83 > TasksTest >