[jira] [Resolved] (KAFKA-15527) Add reverseRange and reverseAll query over kv-store in IQv2

2023-11-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15527. - Resolution: Fixed > Add reverseRange and reverseAll query over kv-store in I

[jira] [Created] (KAFKA-15951) MissingSourceTopicException should include topic names

2023-11-30 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15951: --- Summary: MissingSourceTopicException should include topic names Key: KAFKA-15951 URL: https://issues.apache.org/jira/browse/KAFKA-15951 Project: Kafka

[jira] [Resolved] (KAFKA-15629) proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

2023-11-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15629. - Resolution: Fixed > proposal to introduce IQv2 Query Types: TimestampedKeyQuery

Re: Having a Hard Time Unsubscribing

2023-11-28 Thread Matthias J. Sax
When you send an email to unsubscribe..., there should be a response from the bot. Check your spam folder. You need to ack the unsubscribe by replying the the response email the bot sent. HTH. -Matthias On 11/28/23 6:10 AM, Dasun Nirmitha wrote: Hello Kafka devs, I'm aware that it's a sel

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-21 Thread Matthias J. Sax
Thanks Sophie. Overall I agree with you. I think 50% is too high as a general rule, and I believe something like 30% might be more appropriate (going lower, given the infrastructure at hand might become too aggressive)? The difficult part about a policy like this is, that we don't really hav

Re: [VOTE] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-21 Thread Matthias J. Sax
Matthias J. Sax wrote: Adding an enum is a good idea! Wondering if `UNORDERED` is the best name? Want to avoid bike shedding, just asking. We could also use `UNDEFINED` / `UNSPECIFIED` / `NONE` / `ANY` ? In the end, the result _might_ be ordered, we just don't guarantee any order. -Mat

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-21 Thread Matthias J. Sax
returns an Optional. Cheers, Alieh On Mon, Nov 20, 2023 at 7:38 PM Matthias J. Sax wrote: I think we should also discuss a little more about `validTo()` method? Given that "latest" version does not have a valid-to TS, should we change the return type to `Optional` and return `em

Re: [VOTE] KIP-1000: List Client Metrics Configuration Resources

2023-11-20 Thread Matthias J. Sax
+1 (binding) On 11/20/23 1:42 PM, Jason Gustafson wrote: +1 Thanks for the KIP! On Mon, Nov 20, 2023 at 9:31 AM Jun Rao wrote: Hi, Andrew, Thanks for the KIP. +1 Jun On Thu, Nov 16, 2023 at 9:12 AM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: Hi Apoorv, Thanks for your v

Re: [VOTE] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-20 Thread Matthias J. Sax
Adding an enum is a good idea! Wondering if `UNORDERED` is the best name? Want to avoid bike shedding, just asking. We could also use `UNDEFINED` / `UNSPECIFIED` / `NONE` / `ANY` ? In the end, the result _might_ be ordered, we just don't guarantee any order. -Matthias On 11/20/23 9:17 AM

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-20 Thread Matthias J. Sax
aspects of the KIP are quite clear now and we have come to a consensus about them, I think it 's time to vote ;-) I look forward to your votes. Thanks a lot. Cheers, Alieh On Fri, Nov 17, 2023 at 2:27 AM Matthias J. Sax wrote: Thanks, Alieh. Overall SGTM. About `validTo` -- wondering if we

Re: [DISCUSS] KIP-997 Support fetch(fromKey, toKey, from, to) to WindowRangeQuery and unify WindowKeyQuery and WindowRangeQuery

2023-11-16 Thread Matthias J. Sax
Thanks for the KIP. Given how `WindowRangeQuery` works right now, it's really time to improve it. 1) Agree. It's not clear what will be added right now. I think we should deprecate existing `getKey()` w/o an actually replacement? For `getFromKey` and `getToKey` we should actually be `lowerK

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-16 Thread Matthias J. Sax
e. Having the no-order-guaranteed option does not cost us anything and it keeps the IQv2 interface flexible. I assume we want to drop the Evolving annotation at some point. Sorry for not having brought this up in the discussion about KIP-985. Best, Bruno On 11/15/23 6:56 AM, Matthias J. Sax wr

Re: [VOTE] KIP-997: Partition-Level Throughput Metrics

2023-11-16 Thread Matthias J. Sax
This is KIP-977, right? Not as the subject says. Guess we won't be able to fix this now. Hope it does not cause confusion down the line... -Matthias On 11/16/23 4:43 AM, Kamal Chandraprakash wrote: +1 (non-binding). Thanks for the KIP! On Thu, Nov 16, 2023 at 9:00 AM Satish Duggana wrote:

[jira] [Resolved] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-11-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15346. - Fix Version/s: 3.7.0 Resolution: Fixed > Single-Key_single-timestamp IQs w

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-14 Thread Matthias J. Sax
early a decade and it has never worked. I think we should try something different. When it comes to marking flaky tests as release blockers, I don't think this should be done as a general rule. We should instead assess on a case by case basis, same way we do it for bugs. Ismael On Tue, Nov

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-14 Thread Matthias J. Sax
estamp);", so I feel to be consistent with this API is better compared with being consistent with "WindowKeyQuery"? 3. I agree with Matthias that naming is always tricky, and I also tend to be consistent with what we already have (even if in retro it may not be the best idea :P and if

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-14 Thread Matthias J. Sax
Thanks for starting this discussion David! I totally agree to "no"! I think there is no excuse whatsoever for merging PRs with compilation errors (except an honest mistake for conflicting PRs that got merged interleaved). -- Every committer must(!) check the Jenkins status before merging to av

Re: [VOTE] KIP-998: Give ProducerConfig(props, doLog) constructor protected access

2023-11-07 Thread Matthias J. Sax
avoid exposing our guts to the world (our configuration guts are very, very ugly in my opinion). It will be helpful to end-users as well as plugin authors -- maybe people want this. And it doesn't require the subclassing hack to get access to it. best, Colin On Fri, Nov 3, 2023, at 17:06

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

2023-11-03 Thread Matthias J. Sax
ou mentioned a very important point. How can a user     retrieve the latest value? We have the same issue with kip-968 as well.     Asking a user to call `toTime(max)` violates the API design rules, as you     mentioned. So I think we must have a `latest()` method for both KIP-968 and     KIP-969.

Re: [VOTE] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

2023-11-03 Thread Matthias J. Sax
Thanks for the KIP. +1 (binding) -Matthias On 11/3/23 6:08 AM, Lucas Brutschy wrote: Hi Hanyu, Thanks for the KIP! +1 (binding) Cheers Lucas On Thu, Nov 2, 2023 at 10:19 PM Hao Li wrote: Hi Hanyu, Thanks for the KIP! +1 (non-binding) Hao On Thu, Nov 2, 2023 at 1:29 PM Bill Bejeck wr

Re: [VOTE] KIP-998: Give ProducerConfig(props, doLog) constructor protected access

2023-11-03 Thread Matthias J. Sax
Juma wrote: It seems wrong to require inheritance for this and we already have a public constructor. I would make both of them public. Ismael On Fri, Nov 3, 2023 at 10:47 AM Matthias J. Sax wrote: +1 (binding) About "why not public" question: I think we need to distingu

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

2023-11-03 Thread Matthias J. Sax
you'd like, but hopefully you have no objections to the goal! On Thu, Nov 2, 2023 at 8:44 PM Matthias J. Sax wrote: Almog, can you explain how you intent to implement this change? It's not clear to me, how we could do this? When we call `StreasmBuilder.build()` it will give us a

Re: [VOTE] KIP-998: Give ProducerConfig(props, doLog) constructor protected access

2023-11-03 Thread Matthias J. Sax
+1 (binding) About "why not public" question: I think we need to distinguish between "end users" who create a producer instance, and "external parties" that might implement their own `Producer` (or wrap/extend `KafkaProducer`). In the end, I would not expect an "end user" to actually call `

Re: [DISCUSS] Apache Kafka 3.5.2 release

2023-11-03 Thread Matthias J. Sax
M Matthias J. Sax wrote: Thanks for the info Luke. We did backport all but one PR in the mean time. The missing PR is a RocksDB version bump. We want to consider it for 3.5.2, because it addresses a CVE. Cf https://github.com/apache/kafka/pull/14216 However, RocksDB versions bumps are a littl

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

2023-11-02 Thread Matthias J. Sax
s API already. Last, I am not sure if the new parameter replacing the existing one is the best way to go? I'm happy either way, just let me know which you prefer - the discussion around CUSTOM is in the rejected alternatives but I'm happy to differ to whatever the project

[jira] [Resolved] (KAFKA-15669) Implement telemetry naming strategy

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15669. - Fix Version/s: 3.7.0 Resolution: Fixed > Implement telemetry naming strat

[jira] [Resolved] (KAFKA-15668) Add Opentelemetry Proto library with shadowed classes

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15668. - Fix Version/s: 3.7.0 Resolution: Fixed > Add Opentelemetry Proto library w

Re: ACCESS to Apache Pony Mail

2023-10-31 Thread Matthias J. Sax
Only committers can login using their ASF account. -Matthias On 10/30/23 10:19 PM, Arpit Goyal wrote: Hi Can anyone help me provide access to Apache Pony Mail. I tried login using the jira credential but it didn't work. Thanks and Regards Arpit Goyal 8861094754

[jira] [Created] (KAFKA-15768) StateQueryResult#getOnlyPartitionResult should not throw for FailedQueryResult

2023-10-31 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15768: --- Summary: StateQueryResult#getOnlyPartitionResult should not throw for FailedQueryResult Key: KAFKA-15768 URL: https://issues.apache.org/jira/browse/KAFKA-15768

[jira] [Created] (KAFKA-15765) Remove task level metric "commit-latency"

2023-10-31 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15765: --- Summary: Remove task level metric "commit-latency" Key: KAFKA-15765 URL: https://issues.apache.org/jira/browse/KAFKA-15765 Project: Kafka

[jira] [Resolved] (KAFKA-15672) Add 3.6 to streams system tests

2023-10-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15672. - Resolution: Duplicate > Add 3.6 to streams system te

[jira] [Resolved] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-10-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15602. - Fix Version/s: 3.4.2 3.5.2 3.7.0

[jira] [Reopened] (KAFKA-4852) ByteBufferSerializer not compatible with offsets

2023-10-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-4852: Assignee: (was: LinShunkang) > ByteBufferSerializer not compatible with offs

Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana

2023-10-27 Thread Matthias J. Sax
Congrats! On 10/27/23 10:01 AM, Viktor Somogyi-Vass wrote: Congrats Satish! On Fri, Oct 27, 2023, 18:48 Ivan Yurchenko wrote: Congrats Satish! Ivan On Fri, Oct 27, 2023, at 19:02, Kamal Chandraprakash wrote: Congratulations Satish! On Fri, Oct 27, 2023, 21:10 Jorge Esteban Quilcate Otoya

Re: [DISCUSS] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

2023-10-26 Thread Matthias J. Sax
method names. Sincerely, Hanyu On Fri, Oct 20, 2023 at 10:40 AM Matthias J. Sax wrote: Thanks for the KIP Hanyu, One questions: To address this inconsistency, we propose that KeyQuery should be restricted to querying kv-stores only, ensuring that it always returns a plain

Re: [VOTE] KIP-988 Streams StandbyUpdateListener

2023-10-26 Thread Matthias J. Sax
+1 (binding) On 10/25/23 4:06 PM, Sophie Blee-Goldman wrote: Happy to see this -- that's a +1 (binding) from me On Mon, Oct 23, 2023 at 6:33 AM Bill Bejeck wrote: This is a great addition +1(binding) -Bill On Fri, Oct 20, 2023 at 2:29 PM Almog Gavra wrote: +1 (non-binding) - great impr

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

2023-10-26 Thread Matthias J. Sax
ould even implement this KIP without the end offset and then do a followup KIP to add the end offset within the same release, ie without any deprecations. There are plenty of paths forward here, so don't let us drag this out forever if you know what you want Cheers, Sophie On Fri, Oct

[jira] [Resolved] (KAFKA-15666) Добавить функцию поиска по почте

2023-10-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15666. - Resolution: Invalid [~noraverba] – We can only take tickets in English. – Also piped

[jira] [Created] (KAFKA-15672) Add 3.6 to streams system tests

2023-10-23 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15672: --- Summary: Add 3.6 to streams system tests Key: KAFKA-15672 URL: https://issues.apache.org/jira/browse/KAFKA-15672 Project: Kafka Issue Type: Test

[jira] [Resolved] (KAFKA-15378) Rolling upgrade system tests are failing

2023-10-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15378. - Fix Version/s: 3.4.2 3.5.2 3.7.0

Re: [DISCUSS] Apache Kafka 3.5.2 release

2023-10-20 Thread Matthias J. Sax
Thanks for the info Luke. We did backport all but one PR in the mean time. The missing PR is a RocksDB version bump. We want to consider it for 3.5.2, because it addresses a CVE. Cf https://github.com/apache/kafka/pull/14216 However, RocksDB versions bumps are a little bit more tricky, and w

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

2023-10-20 Thread Matthias J. Sax
Forgot one thing: We could also pass `currentLag()` into `onBachLoaded()` instead of end-offset. -Matthias On 10/20/23 10:56 AM, Matthias J. Sax wrote: Thanks for digging into this Bruno. The JavaDoc on the consumer does not say anything specific about `endOffset` guarantees: Get the

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

2023-10-20 Thread Matthias J. Sax
within the state. That is clearly not the meaning of the event for which the listener should be called. Best, Bruno On 10/19/23 2:12 AM, Matthias J. Sax wrote: Thanks for the KIP. Seems I am almost late to the party. About naming (fun, fun, fun): I like the current proposal overall, except `on

Re: [DISCUSS] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

2023-10-20 Thread Matthias J. Sax
Thanks for the KIP Hanyu, One questions: To address this inconsistency, we propose that KeyQuery should be restricted to querying kv-stores only, ensuring that it always returns a plain V type, making the behavior of the aforementioned code more predictable. Similarly, RangeQuery should

Re: Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2305

2023-10-19 Thread Matthias J. Sax
You would need to unsubscribe from the dev list. I would recommend to setup a filter with you email provider if you don't want these and re-direct them directly to trash. -Matthias On 10/19/23 4:49 AM, Shyam P wrote: how to unsubscribe this ? On Thu, Oct 19, 2023 at 1:30 PM Apache Jenkins

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

2023-10-18 Thread Matthias J. Sax
Thanks for the KIP. Seems I am almost late to the party. About naming (fun, fun, fun): I like the current proposal overall, except `onBachLoaded`, but would prefer `onBatchUpdated`. It better aligns to everything else: - it's an update-listener, not loaded-listener - `StateRestoreListener`

Re: [DISCUSS] Apache Kafka 3.5.2 release

2023-10-17 Thread Matthias J. Sax
Thanks -- there is a few fixed for Kafka Streams we are considering to cherry-pick to get into 3.5.2 release -- what timeline do you target for the release? -Matthias On 10/17/23 8:47 AM, Divij Vaidya wrote: Thank you for volunteering Luke. -- Divij Vaidya On Tue, Oct 17, 2023 at 3:26 PM

Re: [VOTE] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-16 Thread Matthias J. Sax
+1 (binding) On 10/13/23 9:24 AM, Hanyu (Peter) Zheng wrote: Hello everyone, I would like to start a vote for KIP-985 that Add reverseRange and reverseAll query over kv-store in IQv2. Sincerely, Hanyu On Fri, Oct 13, 2023 at 9:15 AM Hanyu (Peter) Zheng wrote: https://cwiki.apache.org/con

Re: Requesting permission for contributions

2023-10-13 Thread Matthias J. Sax
Done. You should be all set. -Matthias On 10/13/23 8:21 AM, Apoorv Mittal wrote: Hi, Can I please get permission to contribute KIP and assign Jiras to myself. Wiki and Jira Id: apoorvmittal10 Email: apoorvmitta...@gmail.com Regards, Apoorv Mittal +44 7721681581

Re: [DISCUSS] 3.5.2 Release

2023-10-13 Thread Matthias J. Sax
Thanks -- there is a few fixed for Kafka Streams we are considering to cherry-pick to get into 3.5.2 release -- can you give us a few more days for this? -Matthias On 10/12/23 6:20 PM, Sophie Blee-Goldman wrote: Thanks for volunteering Luke! On Thu, Oct 12, 2023 at 2:55 AM Levani Kokhreidze

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-13 Thread Matthias J. Sax
javadocs for range() and reverseRange(): range(): *@return The iterator for this range, from smallest to largest bytes.* reverseRange(): * @return The reverse iterator for this range, from largest to smallest key bytes. Cheers, Alieh On Thu, Oct 12, 2023 at 7:32 AM Matthias J. Sax wrote: Quick

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-12 Thread Matthias J. Sax
afka.network:type=RequestMetrics,name=RequestsPerSec,request=*) that report Request rate for every request type? Thanks, Jun On Wed, Oct 11, 2023 at 1:47 PM Matthias J. Sax wrote: Thanks! On 10/10/23 11:31 PM, Andrew Schofield wrote: Matthias, Yes, I think that’s a sensible way forward and the inter

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-12 Thread Matthias J. Sax
r not) could change in the future, but the API can remain the same. Another slight benefit is that there is no need for returning Optional. If the global consumer is not used, it just won't be included in the map. Thanks, Jun On Thu, Oct 12, 2023 at 9:30 AM Matthias J. Sax wrote: Thanks

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-12 Thread Matthias J. Sax
that would apply to the method as a whole. Meanwhile within the method we would issue separate calls to each of the clients using the default or user-configured value of their `default.api.timeout.ms` as the timeout parameter. So the API as proposed makes sense to me. On Wed, Oct 11, 2023 at

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-11 Thread Matthias J. Sax
and I assume also `prefixScan()` (even if there is no "reverse" version for it). On 10/11/23 10:22 PM, Matthias J. Sax wrote: Thanks for raising this question Hanyu. Great find! My interpretation is as follows (it's actually a warning signal that the API contract is not bett

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-11 Thread Matthias J. Sax
nal Bytes to) { if (from != null && to != null && from.compareTo(to) > 0) { This section performs the necessary checks. Sincerely, Hanyu On Thu, Oct 5, 2023 at 9:52 AM Hanyu (Peter) Zheng < pzh...@confluent.io> wrote: Hi, Hao, In this case, it will return

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

2023-10-11 Thread Matthias J. Sax
. Matthias: I do not get what you mean by your second comment. Isn't the KIP already explicit about that? > I assume, results are returned by timestamp for each key. The KIP should be explicit about it. Cheers, Alieh On Tue, Oct 3, 2023 at 6:07 AM Matthias J. Sax wrote: Thank

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-11 Thread Matthias J. Sax
me"     and "fromTime". If I do that, then it is not consistent with KIP-960.     Moreover, the input parameter is clearly a timestamp, which explains     enough. What do you think about that?     5. I was asked to add more examples to the example section. My question     is, wha

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-11 Thread Matthias J. Sax
instances don't return Optional? 132. ClientMetricsSubscriptionRequestCount: Do we need this since we have a set of generic metrics (kafka.network:type=RequestMetrics,name=RequestsPerSec,request=*) that report Request rate for every request type? Thanks, Jun On Wed, Oct 11, 2023 at 1:47 PM Ma

Re: [VOTE] KIP-714: Client metrics and observability

2023-10-11 Thread Matthias J. Sax
+1 (binding) On 9/13/23 5:48 PM, Jason Gustafson wrote: Hey Andrew, +1 on the KIP. For many users of Kafka, it may not be fully understood how much of a challenge client monitoring is. With tens of clients in a cluster, it is already difficult to coordinate metrics collection. When there are th

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-11 Thread Matthias J. Sax
Thanks! On 10/10/23 11:31 PM, Andrew Schofield wrote: Matthias, Yes, I think that’s a sensible way forward and the interface you propose looks good. I’ll update the KIP accordingly. Thanks, Andrew On 10 Oct 2023, at 23:01, Matthias J. Sax wrote: Andrew, yes I would like to get this

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-10 Thread Matthias J. Sax
at can also be included in the proposed Kafka Streams follow-on KIP, or would you prefer that I add it to KIP-714? I have a slight preference for the former to put all of the KS enhancements into a separate KIP. Thanks, Andrew On 7 Oct 2023, at 02:12, Matthias J. Sax wrote: Thanks Andrew. SGTM.

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-09 Thread Matthias J. Sax
t was an accident using getX in the IQv2 KIPs since somebody would have brought it up, otherwise. I am fine with both types of getters. If we think, we need to discuss this in a broader context, let's start a separate thread. Best, Bruno On 10/5/23 7:44 AM, Matthias J. Sax wrote:

Re: [VOTE] KIP-960: Support single-key_single-timestamp interactive queries (IQv2) for versioned state stores

2023-10-09 Thread Matthias J. Sax
One more nit: as discussed on the related KIP-698 thread, we should not use `get` as prefix for the getters. So it should be `K key()` and `Optional asOfTimestamp()`. Otherwise the KIP LGTM. +1 (binding) -Matthias On 10/6/23 2:50 AM, Alieh Saeedi wrote: Hi everyone, Since KIP-960 is red

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-06 Thread Matthias J. Sax
ce and tools do not universally support them. I don’t think the KIP is improved by adding one now. I have also added constants for the various Config classes for ENABLE_METRICS_PUSH_CONFIG, including to StreamsConfig. It’s best to be explicit about this. Thanks, Andrew On 2 Oct 2023, at 23:47, Matt

[jira] [Resolved] (KAFKA-15437) Add metrics about open iterators

2023-10-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15437. - Resolution: Duplicate > Add metrics about open iterat

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-04 Thread Matthias J. Sax
e ending time point of the query, if specified     */    public Optional getAsOfTimestamp(); Other than that I didn't have too much to add. Overall I like the direction of the KIP and think the funcatinlyt is all there! best, Walker On Mon, Oct 2, 2023 at 10:46 PM Matthias J. Sax wr

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-04 Thread Matthias J. Sax
the first key of each. Whether or not this is ordered, could we please add that to the documentation? **(How is this implemented/guaranteed in an `inMemoryKeyValueStore`? I don't know about that implementation). Colt McNealy *Founder, LittleHorse.dev* On Tue, Oct 3, 2023 at 1:35 

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-03 Thread Matthias J. Sax
Thanks for the KIP Hanyu! I took a quick look and it think the proposal makes sense overall. A few comments about how to structure the KIP. As you propose to not add `ReverseRangQuery` class, the code example should go into "Rejected Alternatives" section, not in the "Proposed Changes" secti

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

2023-10-02 Thread Matthias J. Sax
Thanks for updating the KIP. Not sure if I agree or not with Bruno's idea to split the query types further? In the end, we split them only because there is three different return types: single value, value-iterator, key-value-iterator. What do we gain by splitting out single-ts-range-key? In

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-02 Thread Matthias J. Sax
he possibility to make their code more readable instead of only relying on the default. Best, Bruno On 8/17/23 4:13 AM, Matthias J. Sax wrote: Thanks for splitting this part into a separate KIP! For `withKey()` we should be explicit that `null` is not allowed. (Looking into existing `KeyQuery

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-10-02 Thread Matthias J. Sax
ght now, I am composing the integration tests. Let me complete the code and confirm it for 100%. About the KeyQuery class, I thought the KIP must contain just the newly added stuff. OK, I will paste the whole class in KIP-960. Thanks, Alieh On Thu, Aug 17, 2023 at 3:54 AM Matthias J. Sax

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-02 Thread Matthias J. Sax
Hi, I did not pay attention to this KIP in the past; seems it was on-hold for a while. Overall it sounds very useful, and I think we should extend this with a follow up KIP for Kafka Streams. What is unclear to me at this point is the statement: Kafka Streams applications have an applicat

[jira] [Resolved] (KAFKA-15491) RackId doesn't exist error while running WordCountDemo

2023-09-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15491. - Fix Version/s: 3.6.1 3.7.0 Assignee: Hao Li

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

2023-09-29 Thread Matthias J. Sax
Congrats! On 9/25/23 7:29 AM, Rajini Sivaram wrote: Congratulations, Justine! Regards, Rajini On Mon, Sep 25, 2023 at 9:40 AM Lucas Brutschy wrote: Congrats, Justine! On Mon, Sep 25, 2023 at 9:20 AM Bruno Cadonna wrote: Congrats, Justine! Well deserved! Best, Bruno On 9/25/23 5:28 AM

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-29 Thread Matthias J. Sax
Congrats! On 9/26/23 1:30 AM, Mayank Shekhar Narula wrote: Congrats Yash! On Fri, Sep 22, 2023 at 5:24 PM Mickael Maison wrote: Congratulations Yash! On Fri, Sep 22, 2023 at 9:25 AM Chaitanya Mukka wrote: Congrats, Yash!! Well deserved. Chaitanya Mukka On 21 Sep 2023 at 8:58 PM +0530, B

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-29 Thread Matthias J. Sax
Congrats! On 9/26/23 1:29 AM, Mayank Shekhar Narula wrote: Congratulations Lucas! On Fri, Sep 22, 2023 at 5:24 PM Mickael Maison wrote: Congratulations Lucas! On Fri, Sep 22, 2023 at 7:13 AM Luke Chen wrote: Congratulations, Lukas! Luke On Fri, Sep 22, 2023 at 6:53 AM Tom Bentley wrot

[jira] [Created] (KAFKA-15443) Upgrade RocksDB dependency

2023-09-07 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15443: --- Summary: Upgrade RocksDB dependency Key: KAFKA-15443 URL: https://issues.apache.org/jira/browse/KAFKA-15443 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-15437) Add metrics about open iterators

2023-09-05 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15437: --- Summary: Add metrics about open iterators Key: KAFKA-15437 URL: https://issues.apache.org/jira/browse/KAFKA-15437 Project: Kafka Issue Type: New

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-08-16 Thread Matthias J. Sax
Thanks for splitting this part into a separate KIP! For `withKey()` we should be explicit that `null` is not allowed. (Looking into existing `KeyQuery` it seems the JavaDocs don't cover this either -- would you like to do a tiny cleanup PR for this, or fix on-the-side in one of your PRs?)

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-08-16 Thread Matthias J. Sax
tive+queries+%28IQv2%29+for+versioned+state+stores> Cheers, Alieh On Thu, Aug 10, 2023 at 2:38 AM Matthias J. Sax wrote: Seems there was a lot of additional feedback. Looking forward to an updated version of the KIP. I also agree to make the queries more composable. I was considering

[jira] [Resolved] (KAFKA-13197) KStream-GlobalKTable join semantics don't match documentation

2023-08-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13197. - Fix Version/s: 3.6.0 3.5.2 Resolution: Fixed > KStr

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-10 Thread Matthias J. Sax
te. At list with my proposal we could potentially control it with the flag, or maybe create some special aggregate that could be chained after (don't know how to do it yet :-)) Any thoughts? Regards, Igor On Wed, Aug 9, 2023 at 7:19 PM Matthias J. Sax wrote: Thanks for the details

Re: [VOTE] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-10 Thread Matthias J. Sax
+1 (binding) On 8/10/23 12:31 PM, Florin Akermann wrote: https://cwiki.apache.org/confluence/display/KAFKA/KIP-962%3A+Relax+non-null+key+requirement+in+Kafka+Streams

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-10 Thread Matthias J. Sax
n the PR. Great. Yes, in general, I hope everybody agrees that we shouldn't add more details to this KIP On Thu, 10 Aug 2023 at 00:16, Matthias J. Sax wrote: Thanks for the KIP. left join KStream-GlobalTable: no longer drop left records with null-key and call KeyValueMapper with '

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-08-09 Thread Matthias J. Sax
ould not return the lower bound. If I query a version as of a timestamp then the query should return the latest version less than the timestamp. I propose to rename the getters to getTimeFrom() and getTimeTo() as in WindowRangeQuery. 5. Please add the Test Plan section. Regarding long vs Instan

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-09 Thread Matthias J. Sax
do it after. Maybe there could be a flag to stream-table foreign key join that would indicate if we want this join to aggregate children or not? What do you think? Regards, Igor On Fri, Aug 4, 2023 at 10:01 PM Matthias J. Sax wrote: Thanks a lot for providing more background. It's g

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-09 Thread Matthias J. Sax
to XXWithKey in the docs, giving this as one good example that the latter can help you distinguish different scenarios whereas the former cannot. On Fri, Aug 4, 2023 at 6:32 PM Matthias J. Sax wrote: Guozhang, thanks for pointing out ValueJoinerWithKey. In the end, it's just a documentatio

Re: [QUESTION] What is the difference between sequence and offset for a Record?

2023-08-07 Thread Matthias J. Sax
ds. The produce handling on the leader should typically assign the offset. I believe you can find that code here: https://github.com/apache/kafka/blob/b9a45546a7918799b6fb3c0fe63b56f47d8fcba9/core/src/main/scala/kafka/log/UnifiedLog.scala#L766 Justine On Tue, Aug 1, 2023 at 11:38 AM Matthias J. Sax

[jira] [Created] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15309: --- Summary: Add custom error handler to Producer Key: KAFKA-15309 URL: https://issues.apache.org/jira/browse/KAFKA-15309 Project: Kafka Issue Type: New

Re: Testing FixedKeyProcessor implementation using unit tests

2023-08-04 Thread Matthias J. Sax
kafka-repro/blob/master/src/test/java/com/example/demo/MyFixedKeyProcessorTest.java Test is where I am having issues. Thanks, Zed -Original Message- From: Matthias J. Sax Sent: Tuesday, July 11, 2023 1:13 AM To: dev@kafka.apache.org Subject: Re: Testing FixedKeyProcessor implementation

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-04 Thread Matthias J. Sax
JDBC queries from the data source and replace it with CDC streaming data to Kafka. So in that case assembling data entities from Kafka topics instead of JDBC would be beneficial. Please let me know what you think. Regards, Igor On Tue, Jul 25, 2023 at 5:53 PM Matthias J. Sax wrote: Igor, thanks

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-04 Thread Matthias J. Sax
Guozhang, thanks for pointing out ValueJoinerWithKey. In the end, it's just a documentation change, ie, point out that the passed in key could be `null` and similar? -Matthias On 8/2/23 3:20 PM, Guozhang Wang wrote: Thanks Florin for the writeup, One quick thing I'd like to bring up is th

[jira] [Created] (KAFKA-15307) Kafka Streams configuration docs outdate

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15307: --- Summary: Kafka Streams configuration docs outdate Key: KAFKA-15307 URL: https://issues.apache.org/jira/browse/KAFKA-15307 Project: Kafka Issue Type

Re: [QUESTION] What is the difference between sequence and offset for a Record?

2023-08-01 Thread Matthias J. Sax
The _offset_ is the position of the record in the partition. The _sequence number_ is a unique ID that allows broker to de-duplicate messages. It requires the producer to implement the idempotency protocol (part of Kafka transactions); thus, sequence numbers are optional and as long as you don

Re: [VOTE] KIP-759: Unneeded repartition canceling

2023-07-31 Thread Matthias J. Sax
+1 (binding) On 7/11/23 11:16 AM, Shay Lin wrote: Hi all, I'd like to call a vote on KIP-759: Unneeded repartition canceling The KIP has been under discussion for quite some time(two years). This is a valuable optimization for advanced users. I hope we can push this toward the finish line this

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-26 Thread Matthias J. Sax
he only thing I will actually wait on before feeling comfortable enough to vote on this would be a clear method signature (and maybe sample javadocs) in the "Public Interfaces" section. Thanks again for the KIP Shay! Hope I haven't dragged it out too much On Fri, Jul 21

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-07-26 Thread Matthias J. Sax
Thanks for the KIP Alieh. Glad to see that we can add IQ to the new versioned stores! Couple of questions: single-key lookup with timestamp (upper) bound Not sure if "bound" is the right term? In the end, it's a point lookup for a key plus timestamps, so it's an as-of timestamp (not a bou

Re: Streams: Global Store Processors

2023-07-26 Thread Matthias J. Sax
That's correct. And yes, it's a public contract and thus a KIP would be needed to change (or remove it). Deprecation implies that the API is kept for at least 3 releases (ie, one year), plus it can only be removed in a major release. For example, if we deprecated something in 2.1 release, we

[jira] [Created] (KAFKA-15251) Upgrade system test to use 3.5.1

2023-07-25 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15251: --- Summary: Upgrade system test to use 3.5.1 Key: KAFKA-15251 URL: https://issues.apache.org/jira/browse/KAFKA-15251 Project: Kafka Issue Type: Test

<    1   2   3   4   5   6   7   8   9   10   >