Re: [DISCUSS] KIP-1044: A proposal to change idempotent producer -- server implementation

snapshot is significantly reduced. > Significantly >reducing the window where PIDs can be lost. >4. Throttling and other changes you wish to make to the cache are still > possible. > > > On Mon, May 20, 2024 at 7:32 PM Justine Olshan > > wrote: > > > M

[jira] [Resolved] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-16692. Fix Version/s: 3.6.3 Resolution: Fixed > InvalidRequestExcept

Re: [DISCUSS] KIP-1044: A proposal to change idempotent producer -- server implementation

moment? > > > On 20 May 2024, at 18:00, Justine Olshan > wrote: > > > > I would say I have first hand knowledge of this issue as someone who > > responds to such incidents as part of my work at Confluent over the past > > couple years. :) > > > >>

Re: [DISCUSS] KIP-1044: A proposal to change idempotent producer -- server implementation

things. You still have the cache, the point is that it > really is a cache, there is storage behind it. Let the cache be a cache, > let the snapshots be the storage backing behind the cache. > > On Fri, May 17, 2024 at 5:26 PM Justine Olshan > > wrote: > > > Respectfully, I d

Re: [DISCUSS] KIP-1044: A proposal to change idempotent producer -- server implementation

he idea that we can store the hot PIDs in > memory using an LRU and write data to storage so that we can quickly find > things not in the cache is not new. It has been proven. > > I am arguing that we should not throw away state data because we are > running out of memory. We should persis

Re: [DISCUSS] KIP-1046: Expose producer.id.expiration.check.interval.ms as dynamic broker configuration

Hey Jorge, Thanks for the KIP. I didn't realize until I read the details that this configuration is currently not public at all. I think it is still ok that we are exposing the value though. Can we just include some information about the current default, the documentation etc that is already

Re: [DISCUSS] Apache Kafka 3.7.1 release

Hi Igor, +1 from me. Thanks for taking on the release. Justine On Thu, May 16, 2024 at 12:40 AM Chia-Ping Tsai wrote: > +1 > > Our production will be upgraded to your 3.7.1 regardless of how many paper > works I have to handle :) > > Lucas Brutschy 於 2024年5月16日 週四 下午3:25寫道： > > > Hi Igor, >

Re: [DISCUSS] KIP-1044: A proposal to change idempotent producer -- server implementation

+1 to the comment. > I still feel we are doing all of this only because of a few anti-pattern or misconfigured producers and not because we have “too many Producer”. I believe that implementing Producer heartbeat and remove short-lived PIDs from the cache if we didn’t receive heartbeat will be

Re: [VOTE] KIP-1028: Docker Official Image for Apache Kafka

+1 (binding) from me as well. Thanks for working on the KIP. Justine On Tue, May 14, 2024 at 9:20 AM Manikumar wrote: > +1 (binding) > > Thanks for the KIP. > > > > > > On Tue, May 14, 2024, 9:46 PM Chris Egerton > wrote: > > > +1 (binding), thanks for the KIP! > > > > On Tue, May 14, 2024,

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

n, etc.), doesn't that > provide sufficient constraint? I'm not suggesting we eliminate these > methods, just that we change their return types to something more flexible. > > On Mon, May 13, 2024, 12:07 Justine Olshan > wrote: > > > I'm not sure I agree with the Retriable an

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

e of the record upfront is exactly what the KIP proposes? No? > > >> > > >> It achieves the same result but solves it differently, my proposal: > > >> > > >> 1. Application checks the validity of a record (maybe via a new > > >> validateRecord me

[jira] [Resolved] (KAFKA-16513) Allow WriteTxnMarkers API with Alter Cluster Permission

[ https://issues.apache.org/jira/browse/KAFKA-16513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-16513. Resolution: Fixed > Allow WriteTxnMarkers API with Alter Cluster Permiss

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

he message got > > actually > > > produced or not and retrying the message may result in duplicatees. > > > > > >> The thrown TimeoutException "cuts" the connection to the underlying > root > > > cause of missing metadata > > > > > > May

Re: [VOTE] KIP-1041: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)

+1 (binding) Thanks, Justine On Wed, May 8, 2024 at 8:36 AM Federico Valeri wrote: > +1 non binding > > Thanks > > On Wed, May 8, 2024 at 5:27 PM Andrew Schofield > wrote: > > > > Hi, > > Thanks for the KIP. > > > > +1 (non-binding) > > > > Thanks, > > Andrew > > > > > On 8 May 2024, at

Re: Request version not enabled errors upgrading from Kafka 3.5 -> 3.6

Hi Johnson, Thanks for bringing this issue to the mailing list. I'm familiar with the change you are referring to. However, during the upgrade you should be hitting this code path and we should not sending requests to older version brokers.

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

e believe that the user should be able to develop custom exception > > handlers for managing producer exceptions. On the other hand, this will > be > > an expert-level API, and using that may result in strange behaviour in > the > > system, making it hard to find the root cause. Ther

Re: [DISCUSS] KIP-936 Throttle number of active PIDs

the use cases to different KafkaPrincipals or shutdown the misconfigured > app and wait the throttle time pass. The throttle doesn’t get controlled by > the 24hr expiration of PID and we don’t even check if PID has expired or > not. > > I think I might be missing something regard

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

> > then becomes a silent failure that might have repercussions, depending > on > > > the business logic. A user would have to proactively trawls through the > > > logs for WARN/ERROR messages to catch it. > > > > > > Throwing a hard error is prett

Re: [DISCUSS] KIP-936 Throttle number of active PIDs

a quick check on the current CPC to see if it > exceeds hour-limit / 4 and if so check the hour rate (by summing the 4 > 15-minute CPCs). Then the code could simply notify when to throttle and > when to stop throttling. > > Claude > > > https://datasketches.apache.org/docs/CP

Re: [DISCUSS] KIP-936 Throttle number of active PIDs

Hey folks, I shared this with Omnia offline: One concern I have is with the length of time we keep "seen" producer IDs. It seems like the default is 1 hour. If a producer produces every 2 hours or so, and we are hitting the limit, it seems like we will throttle it even though we've seen it before

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

eractions of a producer with > >>>> a ClientExceptionHandler instance will be tricky, and implementing > them > >>>> will also be a fair amount of work. I believe that there needs to be > >> some > >>>> more granularity for how writes to non-existent topic

Re: [VOTE] KIP-1037: Allow WriteTxnMarkers API with Alter Cluster Permission

Hey Nikhil, I meant to comment on the discussion thread, but my draft took so long, you opened the vote. Regardless, I just wanted to say that it makes sense to me. +1 (binding) Justine On Fri, Apr 19, 2024 at 7:22 AM Nikhil Ramakrishnan < ramakrishnan.nik...@gmail.com> wrote: > Hi everyone,

Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

Congratulations Greg! On Thu, Apr 18, 2024 at 12:03 AM Matthias J. Sax wrote: > Congrats Greg! > > On 4/15/24 10:44 AM, Hector Geraldino (BLOOMBERG/ 919 3RD A) wrote: > > Congrats! Well deserved > > > > From: dev@kafka.apache.org At: 04/13/24 14:42:22 UTC-4:00To: > dev@kafka.apache.org > >

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

Hey Alieh, I echo what Omnia says, I'm not sure I understand the implications of the change and I think more detail is needed. This comment also confused me a bit: * {@code ClientExceptionHandler} that continues the transaction even if a record is too large. * Otherwise, it makes the transaction

[jira] [Created] (KAFKA-16570) FenceProducers API returns "unexpected error" when successful

Justine Olshan created KAFKA-16570: -- Summary: FenceProducers API returns "unexpected error" when successful Key: KAFKA-16570 URL: https://issues.apache.org/jira/browse/KAFKA-16570 Proj

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hey folks, Thanks everyone! I will go ahead and call it. The KIP passes with the following +1 votes: - Andrew Schofield (non-binding) - David Jacot (binding) - José Armando García Sancio (binding) - Jun Rao (binding) Thanks again, Justine On Fri, Apr 12, 2024 at 11:16 AM Jun Rao wrote: > Hi,

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

o make it clear that unstable.feature.versions.enable is an > internal configuration. > > Thanks, > > Jun > > On Thu, Apr 11, 2024 at 11:16 AM Justine Olshan > wrote: > > > Updated. :) > > Thanks for the reviews > > > > Justine > > > > On Thu, Apr 11

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

ns. > > Other than that, the KIP looks good to me. > > Jun > > On Thu, Apr 11, 2024 at 9:06 AM Justine Olshan > > wrote: > > > The original config was never actually approved in any KIP. But we can > say > > it is deprecated. > > I can change the config nam

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

arer name? Also, should we > remove/deprecate unstable.metadata.versions.enable in this KIP? > > Jun > > On Tue, Apr 9, 2024 at 9:09 AM Justine Olshan > > wrote: > > > Hi Jun, > > > > Makes sense to me. It seems like KIP-1014 has been inactive re

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

same error `INVALID_UPDATE_VERSION`. I would think continuing to use this error follows naturally, but if we think it is necessary to specify the error code, I can do so in my KIP. Justine On Tue, Apr 9, 2024 at 1:46 PM Justine Olshan wrote: > José, > > INVALID_UPDATE_VERSION was added as part of KIP

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

José, INVALID_UPDATE_VERSION was added as part of KIP-497. The KIP seems to be lacking some details on the error. https://cwiki.apache.org/confluence/display/KAFKA/KIP-497%3A+Add+inter-broker+API+to+alter+ISR https://github.com/apache/kafka/commit/57de67db22eb373f92ec5dd449d317ed2bc8b8d1 The

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

t; > > something is stable in the "this is battle-tested" sense.". I > don't > > >> > think > > >> > > it has to be "battle-tested", it just has to meet the bar of being > > in > > >> the > > >> >

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

consensus with > KIP-1014. > > Thanks, > > Jun > > On Mon, Apr 8, 2024 at 5:06 PM Justine Olshan > > wrote: > > > Hey Jun, > > > > That's a good question. I think maybe for simplicity, we can have a > single > > config? > > If that m

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

. It introduced a new > config unstable.metadata.versions.enable. Does each new feature need to > have a corresponding config to enable the testing of unstable features or > should we have a generic config enabling the testing of all unstable > features? > > Jun > > On Thu

[VOTE] KIP-1022 Formatting and Updating Features

Hello all, I would like to start a vote for KIP-1022 Formatting and Updating Features Please take a look and cast your vote. Thanks, Justine

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

I'm hoping this covers the majority of comments. I will go ahead and open the vote in the next day or so. Thanks, Justine On Wed, Apr 3, 2024 at 3:31 PM Justine Olshan wrote: > Find and replace has failed me :( > > Group version seems a little vague, but we can update it. Hopef

Re: [VOTE] 3.6.2 RC2

will remove the "-SNAPSHOT" suffix while updating the website docs. we > > may need to automate this in the release script. > > > > > > [1] https://github.com/apache/kafka/blob/3.6/docs/documentation.html#L36 > > [2] https://github.com/apache/kafka/blob/3.5/docs

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

t > > transaction version is simpler and conveys more than just protocol > changes > > (flexible records for example) > > > > I will update the KIP as well as KIP-890 > > > > Thanks, > > Justine > > > > On Tue, Apr 2, 2024 at 2:50

Re: [VOTE] 3.6.2 RC2

site-docs artifacts here: > > https://home.apache.org/~manikumar/kafka-3.6.2-rc2/kafka_2.12-3.6.2-site-docs.tgz > These site-docs artifacts will be used to update website docs. > > > Thanks, > > On Wed, Apr 3, 2024 at 10:30 PM Justine Olshan > > wrote: > > > Hi Manikumar, &g

Re: [VOTE] 3.6.2 RC2

Hi Manikumar, I've verified the keys, scanned the artifacts, and other docs. I built from source and ran with a ZK cluster (since I saw that we updated ZK version in this release) I ran a few tests on this cluster. I also ran the 2.12 binary. I noticed the docs link

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

) I will update the KIP as well as KIP-890 Thanks, Justine On Tue, Apr 2, 2024 at 2:50 PM Justine Olshan wrote: > Updated! > > Justine > > On Tue, Apr 2, 2024 at 2:40 PM Jun Rao wrote: > >> Hi, Justine, >> >> Thanks for the reply. >> >>

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

ply. > > > > > > 21. What about the new NextProducerId field? Will that be populated > with > > TV > > > 1? > > > > > > 22. In the dependencies output, should we show both IV and level for > > > metadata.version too? > > > &

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

tine, > > See my comments below. > > On Mon, Apr 1, 2024 at 4:43 PM Justine Olshan > wrote: > > 20. I can update the KIP. > > I took a look at the latest KIP. > > * Should the output of this command "bin/kafka-features.sh > version-mapping --release-versi

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

Rao wrote: > Hi, Justine, > > Thanks for the reply. > > 21. What about the new NextProducerId field? Will that be populated with TV > 1? > > 22. In the dependencies output, should we show both IV and level for > metadata.version too? > > Jun > > On Mon

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

PrevProducerId? But > those fields are only usable after the epoch bump, right? What > functionality does TV 1 achieve? > > Jun > > > On Mon, Apr 1, 2024 at 2:06 PM Justine Olshan > > wrote: > > > I have also updated the KIP to mention the feature tool's --metad

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

I have also updated the KIP to mention the feature tool's --metadata flag will be deprecated. It will still work for users as they learn the new flag, but a warning indicating the alternatives will be shown. Justine On Thu, Mar 28, 2024 at 11:03 AM Justine Olshan wrote: > Hi Jun, > >

[jira] [Resolved] (KAFKA-16451) testDeltaFollower tests failing in ReplicaManager

[ https://issues.apache.org/jira/browse/KAFKA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-16451. Resolution: Duplicate > testDeltaFollower tests failing in ReplicaMana

[jira] [Created] (KAFKA-16451) testDeltaFollower tests failing in ReplicaManager

Justine Olshan created KAFKA-16451: -- Summary: testDeltaFollower tests failing in ReplicaManager Key: KAFKA-16451 URL: https://issues.apache.org/jira/browse/KAFKA-16451 Project: Kafka Issue

Re: [DISCUSS] KIP-890 Server Side Defense

protocol versions included with this KIP. 1 for flexible transaction state records and 2 for KIP-890 part 2 enablement. Justine On Mon, Mar 18, 2024 at 6:39 PM Justine Olshan wrote: > Hey there -- small update to the KIP, > > The KIP mentioned introducing ABORTABLE_ERROR an

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

t; use > > > case is the Queues KIP. I think that we should also use this new flag > to > > > gate it. > > > > > > Best, > > > David > > > > > > On Thu, Mar 28, 2024 at 1:14 AM Jun Rao > > wrote: > > > > > >> Hi,

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

n=2 // Throws error if metadata version is < > > 16, and this would be an upgrade > > But I have updated the KIP to explicitly say the text you mentioned. > > > > Justine > > > > On Wed, Mar 27, 2024 at 1:41 PM José Armando García Sancio > > wrote: &g

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

27, 2024 at 1:41 PM José Armando García Sancio wrote: > Hi Justine, > > See my comment below. > > On Wed, Mar 27, 2024 at 1:31 PM Justine Olshan > wrote: > > The feature command includes the upgrade or downgrade command along with > > the --release-version flag

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

ine On Wed, Mar 27, 2024 at 11:52 AM Jun Rao wrote: > Hi, Justine, > > It seems that we need to specify the dependencies for each feature version? > > Thanks, > > Jun > > On Wed, Mar 27, 2024 at 11:28 AM Justine Olshan > wrote: > > > Hey Jun,

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

, Mar 27, 2024 at 11:14 AM Jun Rao wrote: > Hi, Justine, > > Yes, something like that. We could also extend "kafka-feature describe" by > adding the dependency to every feature in the output. > > Thanks, > > Jun > > On Wed, Mar 27, 2024 at 10:39 AM Justine O

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

lay the commands that would need to be run, if you like. But let > > them decide whether to pull the trigger on each upgrade or downgrade. > > > > This also avoids having to solve the thorny issue of how to have a single > > RPC do both upgrades and downgrades. > > > >

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

García Sancio wrote: > Hi Justine, > > On Mon, Mar 25, 2024 at 5:09 PM Justine Olshan > wrote: > > The reason it is not removed is purely for backwards > > compatibility. Colin had strong feelings about not removing any flags. > > We are not saying that we should remo

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

t? > > 14. The KIP has both transaction.protocol.version vs transaction.version. > What's the correct feature name? > > Jun > > On Mon, Mar 25, 2024 at 4:54 PM Justine Olshan > > wrote: > > > I've updated the KIP to include this CLI. For now, I've moved the new api >

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

), we can have the --release-version flag. I think that closes the main discussions. Please let me know if there is any further discussion I missed. Justine On Mon, Mar 25, 2024 at 4:44 PM Justine Olshan wrote: > Hi Jose, > > Sorry for the typos. I think you figured out what I meant.

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

the update. See my comments below. > > On Mon, Mar 25, 2024 at 2:51 PM Justine Olshan > wrote: > > I've updated the KIP with the changes I mentioned earlier. I have not yet > > removed the --feature-version flag from the upgrade tool. > > What's the "--f

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

, but that didn't seem necessary for this use. Please please try to let me know in the next few days, I hope to start a vote soon. Thanks, Justine On Mon, Mar 18, 2024 at 9:17 AM Justine Olshan wrote: > Hey folks, > > I didn't have a strong preference for requesting the versions via

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

Hey Ahmed, Echoing Andrew's comments to clean up the public interfaces, it was unclear to me if this new flag applies to all the options or just the "latest" option. Clarifying that and showing a few examples could help. Thanks, Justine On Thu, Mar 21, 2024 at 9:43 AM Andrew Schofield <

Re: [DISCUSS] KIP-932: Queues for Kafka

bit more > headroom. Limits like these cannot easily be applied retrospectively, so > the limits are > there right at the start. > > * I have reworked the section on read-committed isolation level, and the > complexity > and memory usage of the approach is significantly improve

Re: [DISCUSS] KIP-890 Server Side Defense

/generic error codes that older clients could support while working on this KIP, so this should help in the future. The features discussion is still ongoing in KIP-1022. Will update again here when that concludes. Justine On Tue, Feb 6, 2024 at 8:39 AM Justine Olshan wrote: > I don't th

Re: [DISCUSS] KIP-932: Queues for Kafka

Hey Andrew, I noticed you started the voting thread, but there seems to be a few questions that were not answered. One was Jun's about memory usage > How much additional heap memory will the server use? Do we need to cache records in heap? If so, is the cache bounded? Your response was > This

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

n Fri, Mar 15, 2024 at 3:38 PM José Armando García Sancio wrote: > Hi Justine, > > Thanks for the update. Some comments below. > > On Wed, Mar 13, 2024 at 10:53 AM Justine Olshan > wrote: > > 4. Include an API to list the features for a given metadata version

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

mply "transaction version" is enough. > > My understanding is that "metadata version" is the version of metadata > records, which is fairly straightforward. "Transaction version" may be > ambiguous. > > -Artem > > On Thu, Feb 29, 2024 at 3:39 PM Justin

Re: [DISCUSS] Apache Kafka 3.6.2 release

Thanks Manikumar! +1 from me Justine On Wed, Mar 13, 2024 at 8:52 AM Manikumar wrote: > Hi, > > I'd like to volunteer to be the release manager for a bug fix release of > the 3.6 line. > If there are no objections, I'll send out the release plan soon. > > Thanks, > Manikumar >

Re: [VOTE] KIP-939: Support Participation in 2PC

Hey Artem, I took another look at the KIP and the discussion on the mailing list. +1 (binding) from me :) Justine On Wed, Mar 6, 2024 at 10:23 AM Martijn Visser wrote: > Hi all, > > It is so exiting to see this KIP and know that it will greatly benefit > Flink and other technologies. > > +1

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

; suggesting that if one uses the tool without either --release-version > > or --features, it will just use the latest version of every feature that > it > > knows. This is what's proposed in the original KIP-584. > > > > 12. If we can't deprecate --metadata MET

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

inue to think about everyone's comments and update the KIP over the next few days. Thanks, Justine On Thu, Feb 29, 2024 at 4:31 PM José Armando García Sancio wrote: > Thanks for the reply Justine. See my comments below: > > On Thu, Feb 29, 2024 at 3:39 PM Justine Olshan > wrote

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

Hey folks, Thanks for the discussion. Let me try to cover everyone's comments. Artem -- I can add the examples you mentioned. As for naming, right now the feature is called "transaction version" or "TV". Are you suggesting it should be called "transaction protocol version" or "TPV"? I don't mind

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

would be helpful to explicit about the syntax for the > enhancements to the tool. I think > that for the features tool, `--release-version` is an optional parameter > which requires a RELEASE_VERSION > argument. I wonder whether it would be helpful to have `--latest-stable` > as an opti

[DISCUSS] KIP-1022 Formatting and Updating Features

Hello folks, I'm proposing a KIP that allows for setting and upgrading new features (other than metadata version) via the kafka storage format and feature tools. This KIP extends on the feature versioning changes introduced by KIP-584 by allowing for the features to be set and upgraded. Please

[jira] [Created] (KAFKA-16308) Formatting and Updating Kafka Features

Justine Olshan created KAFKA-16308: -- Summary: Formatting and Updating Kafka Features Key: KAFKA-16308 URL: https://issues.apache.org/jira/browse/KAFKA-16308 Project: Kafka Issue Type: Task

Re: [DISCUSS] Apache Kafka 3.8.0 release

Thanks Joesp. +1 from me. On Mon, Feb 26, 2024 at 3:37 AM Josep Prat wrote: > Hi all, > > I'd like to volunteer as release manager for the Apache Kafka 3.8.0 > release. > If there are no objections, I'll start building a release plan (or adapting > the one Colin made some weeks ago) in the wiki

Re: [DISCUSS]

Hey there, Thanks for bringing this discussion to the mailing list. I think this is an interesting one. I am little concerned about the ability to add any topic ID to the create topic request. It means that we can no longer rely on topic IDs being globally unique. I worry folks may misunderstand

[jira] [Resolved] (KAFKA-16302) Builds failing due to streams test execution failures

[ https://issues.apache.org/jira/browse/KAFKA-16302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-16302. Resolution: Fixed > Builds failing due to streams test execution failu

[jira] [Created] (KAFKA-16302) Builds failing due to streams test execution failures

Justine Olshan created KAFKA-16302: -- Summary: Builds failing due to streams test execution failures Key: KAFKA-16302 URL: https://issues.apache.org/jira/browse/KAFKA-16302 Project: Kafka

[jira] [Resolved] (KAFKA-15665) Enforce ISR to have all target replicas when complete partition reassignment

[ https://issues.apache.org/jira/browse/KAFKA-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15665. Resolution: Fixed > Enforce ISR to have all target replicas when complete partit

Re: [VOTE] 3.7.0 RC4

Hey folks, I've done the following to validate the release: -- validated the keys for all the artifacts -- built from source and started a ZK cluster -- ran a few workloads on it. -- ran 2.12 Kraft cluster and ran a few workloads on it I see there is a lot of ongoing discussion about the

Re: [VOTE] 3.7.0 RC4

FailureTest.testProduceErrorFromFailureOnLogRoll > - kafka.server.LogDirFailureTest.testIOExceptionDuringLogRoll > failed three timed in a row. > > Here is the gist with the results: > https://gist.github.com/stanislavkozlovski/820976fc7bfb5f4dcdf9742fd96a9982 > > > > On Wed, Feb 14, 2024 at 6:31 PM Justine Olshan

Re: [VOTE] 3.7.0 RC4

Hey Stan, Did we ever get system tests results? I can also start a run On Mon, Feb 12, 2024 at 1:06 PM Jakub Scholz wrote: > +1 (non-binding). I used the staged binaries with Scala 2.13 and the staged > Maven artifacts to run my tests. All seems to work fine. Thanks. > > Jakub > > On Fri, Feb

[jira] [Created] (KAFKA-16245) DescribeConsumerGroupTest failing

Justine Olshan created KAFKA-16245: -- Summary: DescribeConsumerGroupTest failing Key: KAFKA-16245 URL: https://issues.apache.org/jira/browse/KAFKA-16245 Project: Kafka Issue Type: Task

[jira] [Resolved] (KAFKA-16229) Slow expiration of Producer IDs leading to high CPU usage

[ https://issues.apache.org/jira/browse/KAFKA-16229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-16229. Resolution: Fixed > Slow expiration of Producer IDs leading to high CPU us

Re: [DISCUSS] KIP-890 Server Side Defense

ion be gated by a new MV? If so, does > downgrading a TV imply potential downgrade of MV too? > > Jun > > > > On Mon, Feb 5, 2024 at 5:07 PM Justine Olshan > > wrote: > > > One TV gates the flexible feature version (no rpcs involved, only the > > transacti

Re: [DISCUSS] KIP-890 Server Side Defense

TV guard? Going forward, will those > PRC/record protocols only be guarded by TV and not by other features like > MV? > > Thanks, > > Jun > > On Mon, Feb 5, 2024 at 2:41 PM Justine Olshan > > wrote: > > > Hi Jun, > > > > Sorry I think I m

Re: [DISCUSS] KIP-890 Server Side Defense

Thanks, > > Jun > > On Fri, Feb 2, 2024 at 1:06 PM Justine Olshan > > wrote: > > > Hey Jun, > > > > Yes, the idea is that if we downgrade TV (transaction version) we will > stop > > using the add partitions to txn optimization and stop writing the &g

Re: [DISCUSS] KIP-890 Server Side Defense

automatically downgraded too? > > Jun > > On Tue, Jan 30, 2024 at 3:32 PM Justine Olshan > > wrote: > > > Hey Jun, > > > > I wanted to get back to you about your questions about MV/IBP. > > > > Looking at the options, I think it makes the most sens

Re: [VOTE] KIP-974: Docker Image for GraalVM based Native Kafka Broker

Hi Krishna, Thanks for following up on my questions in the discussion thread. +1 (binding) from me :) Justine On Mon, Jan 29, 2024 at 9:00 AM Krishna Agarwal < krishna0608agar...@gmail.com> wrote: > Hi, > A gentle nudge to cast your vote on the KIP. > Your input is highly valued. > > Regards, >

Re: [DISCUSS] KIP-974 Docker Image for GraalVM based Native Kafka Broker

by using the Alpine image. Should future instances suggest the > need for a different Docker base image for optimisation, we can explore > alternatives at that time. > > Regards, > Krishna > > On Thu, Feb 1, 2024 at 12:32 AM Justine Olshan > > wrote: > > > Hey Kr

Re: [DISCUSS] KIP-974 Docker Image for GraalVM based Native Kafka Broker

Hey Krishna, Can we include the perf results between the distroless/alpine/ubuntu images in the KIP? I also noticed - Alpine employs the apk package manager, which, being relatively less popular, may pose challenges in the future. There's a potential risk that certain libraries we

Re: [DISCUSS] KIP-890 Server Side Defense

protocol) I've updated the KIP to include this change. Hopefully that's everything we need for this KIP :) Justine On Mon, Jan 22, 2024 at 3:17 PM Justine Olshan wrote: > Thanks Jun, > > I will update the KIP with the prev field for prepare as well. > > PREPARE > produc

Re: Broken build

ine, > > Thanks for stopping the build! > I was using the CloudBees UI to stop the build and it wasn't working, > thanks for pointing out that the Jenkins button works. I'll keep that > in mind for next time :) > > Greg > > On Tue, Jan 30, 2024 at 1:27 PM Justine Ols

Re: Broken build

Hey Greg, I found a bunch of builds stuck like this last week. You have to click in the console window for force terminate. I forced the termination as I did with the others. Justine On Tue, Jan 30, 2024 at 12:21 PM Greg Harris wrote: > Hey Josep, > > Thanks for finding and fixing this build

[jira] [Resolved] (KAFKA-16122) TransactionsBounceTest -- server disconnected before response was received

[ https://issues.apache.org/jira/browse/KAFKA-16122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-16122. Resolution: Fixed > TransactionsBounceTest -- server disconnected before respo

[jira] [Resolved] (KAFKA-15987) Refactor ReplicaManager code for transaction verification

[ https://issues.apache.org/jira/browse/KAFKA-15987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15987. Resolution: Fixed > Refactor ReplicaManager code for transaction verificat

Re: Kafka trunk test & build stability

believe the rest are running as expected now. Justine On Thu, Jan 25, 2024 at 10:22 AM Justine Olshan wrote: > Hey folks -- I noticed some builds have been running for a day or more. I > thought we limited builds to 8 hours. Any ideas why this is happening? > > > https://ci-builds.

Re: Kafka trunk test & build stability

gt; > > > >>> Thanks for all the work that has already been done on this in the > > past > > > >>> days! > > > >>> > > > >>> Have we considered running our test suite with > > > >>> -XX:+HeapDumpOnOutOfMemoryError and uploading the h

[jira] [Created] (KAFKA-16192) Introduce usage of flexible records to coordinators

Justine Olshan created KAFKA-16192: -- Summary: Introduce usage of flexible records to coordinators Key: KAFKA-16192 URL: https://issues.apache.org/jira/browse/KAFKA-16192 Project: Kafka

Re: [VOTE] 3.7.0 RC2

Oops sorry I got confused between https://issues.apache.org/jira/browse/KAFKA-16120 (migration issue) and https://issues.apache.org/jira/browse/KAFKA-14616 (not migration issue) However, both do not seem related to JBOD based on the jira and PRs Justine On Tue, Jan 23, 2024 at 1:51 PM Justine

  1   2   3   4   5   >