[jira] [Resolved] (KAFKA-16349) ShutdownableThread fails build by calling Exit with race condition

[ https://issues.apache.org/jira/browse/KAFKA-16349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-16349. - Fix Version/s: 3.8.0 Assignee: Greg Harris Resolution: Fixed >

Re: [VOTE] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

+1 Not binding :) On Mon, Mar 25, 2024 at 11:11 AM Walker Carlson wrote: > Hello everybody, > > I think we have had some pretty good discussion on this kip and it seems > that we are close if not yet settled on the final version. > > So I would like to open up the voting for KIP-1024: >

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.7 #122

See

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2763

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.5 #106

See

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Jun, thanks a lot for your help. I feel that the KIP is much better after your detailed input. If there is no more feedback, I'll start a voting thread tomorrow morning. I'll monitor KIP-1022's discussion thread and update this KIP with anything that affects the KIP's specification. Thanks, --

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Hi, Jose, Thanks for the explanation. Other than depending on KIP-1022 to be approved, the KIP looks good to me now. Jun On Thu, Mar 28, 2024 at 2:56 PM José Armando García Sancio wrote: > Hi Jun, > > See my comments below. > > On Thu, Mar 28, 2024 at 11:09 AM Jun Rao wrote: > > If I am

[jira] [Resolved] (KAFKA-16286) KRaft doesn't always notify listener of latest leader

[ https://issues.apache.org/jira/browse/KAFKA-16286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio resolved KAFKA-16286. Resolution: Fixed > KRaft doesn't always notify listener of

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Hi Jun, See my comments below. On Thu, Mar 28, 2024 at 11:09 AM Jun Rao wrote: > If I am adding a new voter and it takes a long time (because the new voter > is catching up), I'd want to know if the request is indeed being processed. > I thought that's the usage of uncommitted-voter-change.

Re: [DISCUSS] KIP-890 Server Side Defense

Hi there -- another update! When looking into the implementation for the safe epoch bumps I realized that we are already populating previousProducerID in memory as part of KIP-360. If we are to start using flexible fields, it is better to always use this information and have an explicit (tagged)

[jira] [Created] (KAFKA-16446) Log slow controller events

David Arthur created KAFKA-16446: Summary: Log slow controller events Key: KAFKA-16446 URL: https://issues.apache.org/jira/browse/KAFKA-16446 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-477: Add PATCH method for connector config in Connect REST API

 No further comments from me. On Thu, Mar 28, 2024, 19:36 Ivan Yurchenko wrote: > Hello Chris, > > Thanks for your feedback. I created the jira and also updated the > description a bit mentioning that other similar race scenarios exist and > the KIP is not trying to solve them. > > Best, >

Re: [DISCUSS] KIP-477: Add PATCH method for connector config in Connect REST API

Hello Chris, Thanks for your feedback. I created the jira and also updated the description a bit mentioning that other similar race scenarios exist and the KIP is not trying to solve them. Best, Ivan On Wed, Mar 27, 2024, at 17:08, Chris Egerton wrote: > Hi Ivan, > > Thanks for the updates.

[jira] [Created] (KAFKA-16445) PATCH method for connecto configuration

Ivan Yurchenko created KAFKA-16445: -- Summary: PATCH method for connecto configuration Key: KAFKA-16445 URL: https://issues.apache.org/jira/browse/KAFKA-16445 Project: Kafka Issue Type:

[PR] MINOR: update 3.6 [kafka-site]

wcarlson5 opened a new pull request, #593: URL: https://github.com/apache/kafka-site/pull/593 Take the changes from https://github.com/apache/kafka/pull/14725/files and make them live for 3.6 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Hi, Jose, Thanks for the reply. If I am adding a new voter and it takes a long time (because the new voter is catching up), I'd want to know if the request is indeed being processed. I thought that's the usage of uncommitted-voter-change. Also, I am still not sure about having multiple brokers

[jira] [Created] (KAFKA-16444) Run KIP-848 unit tests under code coverage

Kirk True created KAFKA-16444: - Summary: Run KIP-848 unit tests under code coverage Key: KAFKA-16444 URL: https://issues.apache.org/jira/browse/KAFKA-16444 Project: Kafka Issue Type: Task

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

Hi Jun, For both transaction state and group coordinator state, there are only version 0 records. KIP-915 introduced flexible versions, but it was never put to use. MV has never gated these. This KIP will do that. I can include this context in the KIP. I'm happy to modify his 1 and 2 to 0 and 1.

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

Hi, David, Thanks for the reply. Historically, the format of all records were controlled by MV. Now, records in _offset_commit will be controlled by `group.coordinator.version`, is that right? It would be useful to document that. Also, we should align on the version numbering. "kafka-feature

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Hi Jun, On Thu, Mar 28, 2024 at 10:35 AM Jun Rao wrote: > The following are the steps of AddVoter. The bulk of the time is probably > in step 5, but the updated VotersRecord won't be written until step 6. So, > ideally, the controller leader should report the pending voter as soon as > step 1.

[jira] [Created] (KAFKA-16443) Update streams_static_membership_test.py to support KIP-848’s group protocol config

Kirk True created KAFKA-16443: - Summary: Update streams_static_membership_test.py to support KIP-848’s group protocol config Key: KAFKA-16443 URL: https://issues.apache.org/jira/browse/KAFKA-16443

[jira] [Created] (KAFKA-16442) Update streams_standby_replica_test.py to support KIP-848’s group protocol config

Kirk True created KAFKA-16442: - Summary: Update streams_standby_replica_test.py to support KIP-848’s group protocol config Key: KAFKA-16442 URL: https://issues.apache.org/jira/browse/KAFKA-16442 Project:

[jira] [Created] (KAFKA-16440) Update security_test.py to support KIP-848’s group protocol config

Kirk True created KAFKA-16440: - Summary: Update security_test.py to support KIP-848’s group protocol config Key: KAFKA-16440 URL: https://issues.apache.org/jira/browse/KAFKA-16440 Project: Kafka

[jira] [Created] (KAFKA-16441) Update streams_broker_down_resilience_test.py to support KIP-848’s group protocol config

Kirk True created KAFKA-16441: - Summary: Update streams_broker_down_resilience_test.py to support KIP-848’s group protocol config Key: KAFKA-16441 URL: https://issues.apache.org/jira/browse/KAFKA-16441

[jira] [Created] (KAFKA-16439) Update replication_replica_failure_test.py to support KIP-848’s group protocol config

Kirk True created KAFKA-16439: - Summary: Update replication_replica_failure_test.py to support KIP-848’s group protocol config Key: KAFKA-16439 URL: https://issues.apache.org/jira/browse/KAFKA-16439

[jira] [Created] (KAFKA-16438) Update consumer_test.py’s static tests to support KIP-848’s group protocol config

Kirk True created KAFKA-16438: - Summary: Update consumer_test.py’s static tests to support KIP-848’s group protocol config Key: KAFKA-16438 URL: https://issues.apache.org/jira/browse/KAFKA-16438 Project:

[jira] [Resolved] (KAFKA-16271) Update consumer_rolling_upgrade_test.py to support KIP-848’s group protocol config

[ https://issues.apache.org/jira/browse/KAFKA-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True resolved KAFKA-16271. --- Reviewer: Lucas Brutschy Resolution: Fixed > Update consumer_rolling_upgrade_test.py to

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Hi, Jose, Thanks for the reply. The following are the steps of AddVoter. The bulk of the time is probably in step 5, but the updated VotersRecord won't be written until step 6. So, ideally, the controller leader should report the pending voter as soon as step 1. The brokers and non-leader

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

Hey all, Thanks for the feedback Bruno, Almog and Matthias! Almog: I like the idea, but I agree with Matthais. I actually looked at that ticket a bit when doing this and found that while similar they are actually pretty unrelated codewise. I would love to see it get taken care of. Bruno and

Re: [DISCUSS] KIP-816: Topology changes without local state reset

Hi everyone, I'm going to resurrect this KIP, because I would like the community to benefit from our solution. In the end, we internally solved this problem using Option B: automatically moving state directories to the correct location whenever they're no longer aligned with the Topology. We

Re: [DISCUSS] KIP-989: RocksDB Iterator Metrics

Quick addendum: My suggested metric "oldest-open-iterator-age-seconds" should be "oldest-open-iterator-age-ms". Milliseconds is obviously a better granularity for such a metric. Still accepting suggestions for a better name. On Thu, 28 Mar 2024 at 13:41, Nick Telford wrote: > Hi everyone, > >

Re: [DISCUSS] KIP-989: RocksDB Iterator Metrics

Hi everyone, Sorry for leaving this for so long. So much for "3 weeks until KIP freeze"! On Sophie's comments: 1. Would Matthias's suggestion of a separate metric tracking the age of the oldest open iterator (within the tag set) satisfy this? That way we can keep iterator-duration-(avg|max) for

Re: [DISCUSS] KIP-990: Capability to SUSPEND Tasks on DeserializationException

Hi folks, Sorry I haven't got back to you until now. It's become clear that I hadn't anticipated a significant number of technical challenges that this KIP presents. I think expecting users to understand the ramifications on aggregations, joins and windowing ultimately kills it: it only becomes

Assistance Needed with Creating Wiki ID for Kafka Contribution

Hi, I hope this email finds you well. I've been eager to contribute to Apache Kafka and have started following the procedure outlined in the Kafka Improvement Proposals (KIP) documentation -

Re: [DISCUSS] KIP-1027 Add MockFixedKeyProcessorContext

It seems that `MockRecordMetadata` is a private class, and thus not part of the public API. If there are any changes required, we don't need to discuss on the KIP. For `CapturedPunctuator` and `CapturedForward` it's a little bit more tricky. My gut feeling is, that the classes might not need

Re: [DISCUSS] KIP-1020: Deprecate `window.size.ms` in `StreamsConfig`

Thanks. I think you can start a VOTE. -Matthias On 3/20/24 9:24 PM, Lucia Cerchie wrote: thanks Sophie, I've made the updates, would appreciate one more look before submission On Wed, Mar 20, 2024 at 8:36 AM Sophie Blee-Goldman wrote: A few minor notes on the KIP but otherwise I think you

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

Hey, looking into the API, I am wondering why we would need to add an overload talking a `Named` parameter? StreamsBuilder.addGlobalStore() (and .addGlobalTable()) already takes a `Consumed` parameter that allows to set a name. 2. I do not understand what you mean with "maximum

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

Hi Satish, I will try to answer as much as I can and the others could chime in with further details. *101. For remote.log.disable.policy=delete: Does it delete the remote log data immediately and the data in remote storage will not be taken into account by any replica? That means

[VOTE] 3.6.2 RC2

Hello Kafka users, developers and client-developers, This is the second candidate we are considering for the release of Apache Kafka 3.6.2. This is a bugfix release with several fixes, including dependency version bumps for CVEs. Release notes for the 3.6.2 release:

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

Hi David, I agree that we should use the same mechanism to gate KIP-932 once that feature reaches production readiness. The precise details of the values will depend upon the current state of all these flags when that release comes. Thanks, Andrew > On 28 Mar 2024, at 07:11, David Jacot wrote:

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.6 #167

See

Re: [ANNOUNCE] New committer: Christo Lolov

Congratulations! On Wed, 27 Mar 2024 at 21:10, Matthias J. Sax wrote: > Congrats! > > On 3/26/24 9:39 PM, Christo Lolov wrote: > > Thank you everyone! > > > > It wouldn't have been possible without quite a lot of reviews and > extremely > > helpful inputs from you and the rest of the community!

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

Hi, Jun, Justine, Regarding `group.coordinator.version`, the idea is to use it to gate records and APIs of the group coordinator. The first use case will be KIP-848. We will use version 2 of the flag to gate all the new records and the new ConsumerGroupHeartbeat/Describe APIs present in AK 3.8.