[jira] [Resolved] (KAFKA-13654) Extend KStream process with new Processor API

2022-04-19 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13654. -- Fix Version/s: 3.3.0 Resolution: Fixed > Extend KStream process with new Processor

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

2022-04-15 Thread John Roesler
Hi Chaimaa, Interesting... That basis for those metrics is the difference between a record's timestamp and the current system time while processing it. I wonder if it's negative because of a difference in the way that the producer sets the records' timestamps versus the current time on the

Re: [DISCUSS] KIP-829: (console-consumer) add print.topic property

2022-04-15 Thread John Roesler
Hi Evans, Thanks for the KIP! I agree with Mickael; it would be good to see an example in the KIP. Otherwise, I'm in favor. Thanks, -John On Fri, Apr 15, 2022, at 04:33, Mickael Maison wrote: > Hi, > > I can't believe this was missing! Thanks for addressing this. > Can you add an example of

Re: [VOTE] KIP-813 Shared State Stores

2022-04-14 Thread John Roesler
up on the discussion thread and I've gone over the KIP. This > seems like a good addition to me. > > +1 (binding) > > Thanks, > Bill > > On Fri, Apr 1, 2022 at 2:13 PM Matthias J. Sax wrote: > >> +1 (binding) >> >> >> On 4/1/22 6:47 AM, John R

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-04-11 Thread John Roesler
ntly it doesn't work... >> >>>>> >> >>>>> On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna >> >>>> wrote: >> >>>>> >> >>>>>> Hi Hao, >> >>>>>> >> >>>>>>

[jira] [Created] (KAFKA-13820) Add debug-level logs to explain why a store is filtered out during interactive query

2022-04-11 Thread John Roesler (Jira)
John Roesler created KAFKA-13820: Summary: Add debug-level logs to explain why a store is filtered out during interactive query Key: KAFKA-13820 URL: https://issues.apache.org/jira/browse/KAFKA-13820

[jira] [Created] (KAFKA-13819) Add application.server to Streams assignor logs when set

2022-04-11 Thread John Roesler (Jira)
John Roesler created KAFKA-13819: Summary: Add application.server to Streams assignor logs when set Key: KAFKA-13819 URL: https://issues.apache.org/jira/browse/KAFKA-13819 Project: Kafka

[jira] [Created] (KAFKA-13818) Add generation to consumer assignor logs

2022-04-11 Thread John Roesler (Jira)
John Roesler created KAFKA-13818: Summary: Add generation to consumer assignor logs Key: KAFKA-13818 URL: https://issues.apache.org/jira/browse/KAFKA-13818 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-13479) Interactive Query v2

2022-04-06 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13479. -- Fix Version/s: 3.2.0 Resolution: Fixed > Interactive Query

Re: Kafka Streams Issue

2022-04-01 Thread John Roesler
Hi Daan, First of all, it does sound like that is a correct implementation of QueryableStoreProvider. Kudos for taking that on; the complexity of that API was one of my top motivations for replacing it with IQv2! (https://cwiki.apache.org/confluence/display/KAFKA/KIP-796%3A+Interactive+Query+v2 )

Re: [VOTE] KIP-813 Shared State Stores

2022-04-01 Thread John Roesler
Thanks for the KIP, Daan! I’m +1 (binding) -John On Tue, Mar 29, 2022, at 06:01, Daan Gertis wrote: > I would like to start a vote on this one: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-813%3A+Shareable+State+Stores > > Cheers, > D.

Re: [DISCUSS] KIP-813 Shared State Stores

2022-04-01 Thread John Roesler
restoration latency during failover) would not be served. > > Yeah I think standbys should be applicable here as well. But we get > that by implementing these readonly statestores as regular ones right? > > Cheers, > D. > > > From: John Roesler > Date: Friday, 1

Re: [DISCUSS] KIP-813 Shared State Stores

2022-03-31 Thread John Roesler
Hi Daan, Thanks for the KIP! I just got caught up on the discussion. I just have a some small questions, and then I will be ready to vote. 1. Am I right I’m thinking that there’s no way to enforce the stores are actually read-only, right? It seems like the StoreBuilder interface is too

Re: [VOTE] KIP-824 Allowing dumping segmentlogs limiting the batches in the output

2022-03-26 Thread John Roesler
Thanks for the KIP, Sergio! I’m +1 (binding) Thanks, John On Sat, Mar 26, 2022, at 03:32, David Jacot wrote: > +1 (binding). Thanks for the KIP. > > Best, > David > > Le ven. 25 mars 2022 à 07:11, Luke Chen a écrit : > >> Hi Sergio, >> >> Thanks for the KIP! >> +1(binding) from me. >> >> Thank

[jira] [Resolved] (KAFKA-13554) Rename RangeQuery to KeyRangeQuery

2022-03-24 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13554. -- Resolution: Won't Fix > Rename RangeQuery to KeyRangeQu

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
Thanks, Hao, I’m +1 (binding) -John On Thu, Mar 24, 2022, at 11:38, Hao Li wrote: > Hi all, > > I'd like to start a vote on Kafka Streams KIP-825: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced > > Thanks, > Hao

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
tting of results be > > > > > controlled by the window level api? If I want to emit results for each > > > > > input record the emit strategy is quite independent from the window. > > > > > So > > > > > I somehow share Matthias' and Guozhang's conc

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
Thanks, Hao! I'm +1 (binding) -John On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote: > Hi all, > > I'd like to start a vote on Kafka Streams KIP-825: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced

[jira] [Resolved] (KAFKA-13714) Flaky test IQv2StoreIntegrationTest

2022-03-23 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13714. -- Fix Version/s: 3.2.0 Assignee: John Roesler Resolution: Fixed > Flaky t

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-23 Thread John Roesler
> example above are operations on data. `groupBy()` groups, > > > > > `windowedBy()` > > > > > > > > partitions, `aggregate()` computes the aggregate, `mapValues()` > > > > > > > > maps

Re: [VOTE} KIP-796: Interactive Query v2

2022-03-22 Thread John Roesler
released for the first time, but after some second thoughts I'm now >> feeling it's okay to take it in a more evolving manner. So I'm preferring >> to keep it and be open for breaking changes in the future. >> >> >> Guozhang >> >> On Tue, Mar 22, 2022 at

Re: [VOTE} KIP-796: Interactive Query v2

2022-03-22 Thread John Roesler
xisting query types specified in KIP-805 and > KIP-806? > > BTW, I now realize that I mistakenly removed KIP-796 from the release > plan for 3.2. Sorry for that! I will re-add it to the release plan. > > Best, > Bruno > > On 22.03.22 02:50, John Roesler wrote: > > He

Re: [VOTE} KIP-796: Interactive Query v2

2022-03-21 Thread John Roesler
something later. What is your preference? Release it, or wait? Thanks, John On Mon, Nov 22, 2021, at 21:18, John Roesler wrote: > Thanks for voting and for the discussion, all! > > The vote on KIP-796 passes with: > 3 binding +1 (Bruno, Bill, and myself) > 2 non-binding +1 (Patri

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-21 Thread John Roesler
es covered by Suppressed today, we can discuss if we could > have `KTable.suppress(Emitted..)` replacing `KTable.suppress(Suppressed..)` > as a whole, but for this KIP I think it's too early. > > > Guozhang > > > On Mon, Mar 21, 2022 at 6:18 PM John Roesler wrote: > >

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-21 Thread John Roesler
Hi all, Thanks for the Kip, Hao! For what it’s worth, I’m also in favor of your latest framing of the API, I think the name is fine. I assume it’s inspired by Flink? It’s not identical to the concept of a trigger in Flink, which specifies when to evaluate the window, which might be confusing

Re: [VOTE] KIP-820: Extend KStream process with new Processor API

2022-03-15 Thread John Roesler
hey also include `final String... stateStoreNames`. > And in javadocs explains that if users want to connect state stores to this > processor, they could use the `connectState` API instead. > > Otherwise, I'm +1. > > Guozhang > > On Tue, Feb 15, 2022 at 11:54 AM John Roe

[jira] [Created] (KAFKA-13714) Flaky test IQv2StoreIntegrationTest

2022-03-07 Thread John Roesler (Jira)
John Roesler created KAFKA-13714: Summary: Flaky test IQv2StoreIntegrationTest Key: KAFKA-13714 URL: https://issues.apache.org/jira/browse/KAFKA-13714 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-03-06 Thread John Roesler
belatedly was that if we do set it to > > > > > > Void, then users will actually have to override the key when > > > > > > forwarding, like `record.withKey(null)`, whereas if we keep > > > > > > it is K, all users have to do is not touch the key at all. &

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-25 Thread John Roesler
at 11:07 -0600, John Roesler wrote: > Hello all, > > I'll chime in again in the interest of trying to do a better > job of keeping KIPs moving forward... > > Matthias raised some very good questions about whether the > change is really source compatible. I just checked ou

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-25 Thread John Roesler
check could be sufficient; as for the > > purpose of 2), as for this KIP itself I think it is similar to what we have > > (i.e. just base on the function name "processValue" itself) and hence are > > not sacrificed either. I do not know if > > `KStream#processValu

Re: [VOTE] KIP-815: Support max-timestamp in GetOffsetShell

2022-02-23 Thread John Roesler
Thanks for the KIP, Ziming! I'm +1 (binding) -John On Wed, 2022-02-23 at 13:54 +0100, Mickael Maison wrote: > Hi, > > I'm +1 (binding) too. > > Just one minor comment: > Could we make the "time" argument also accept "earliest", "latest" and > "max-timestamp" alongside -1, -2, -3. I think it's

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-18 Thread John Roesler
specific >> type >> >> would break binary compatibility. >> >> I will update the KIP to reflect this: >> >> >> >>> Modifications to method KStream#process are source compatible with >> >> previous version, though not binary compat

Re: [VOTE] KIP-812: Introduce another form of the `KafkaStreams.close()` API that forces the member to leave the consumer group

2022-02-17 Thread John Roesler
Thanks for the KIP, Sueng-Chan! I’m +1 (binding) -John On Thu, Feb 17, 2022, at 12:58, Guozhang Wang wrote: > Thanks Sueng-chan, > > I'm +1 on the proposal. > > > Guozhang > > On Tue, Feb 15, 2022 at 7:06 AM Seung-chan Ahn > wrote: > >> Hi team, >> >> I feel like we have a finalized proposal

Re: [DISCUSS] KIP-634: Complementary support for headers in Kafka Streams DSL

2022-02-15 Thread John Roesler
citly. However, it > > would be good to state whether and when we'll automatically > > wrap a value serde. For example, if the value serde is known > > (or if we're using a default serde from the config), will > > Streams automatically wrap it downstream of the record- > > map

Re: [VOTE] KIP-820: Extend KStream process with new Processor API

2022-02-15 Thread John Roesler
Thanks, Jorge! I'm +1 (binding) -John On Tue, 2022-02-15 at 19:16 +, Jorge Esteban Quilcate Otoya wrote: > Hi all, > > I'd like to start a vote on KIP-820 which proposes extending KStream to use > the new Processor API >

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-15 Thread John Roesler
My apologies, this feedback was intended for KIP-634. -John On Tue, Feb 15, 2022, at 13:15, John Roesler wrote: > Thanks for the update, Jorge, > > I've just looked over the KIP again. Just one more small > concern: > > 5) We can't just change the type of Record#headers(

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-15 Thread John Roesler
clude the deprecation of: > > > >- KStream#transform > >- KStream#transformValues > >- KStream#flatTransform > >- KStream#flatTransformValues > > > > > > > > On Fri, 11 Feb 2022 at 15:16, John Roesler wrote: > > > >

Re: [DISCUSS] KIP-816: Topology changes without local state reset

2022-02-14 Thread John Roesler
e old > > > topology's sub-topology 0' state store A-0004, then what we can do to let > > > the new topology state store to be loaded as the old state store. With > > that > > > in my mind originally, I said maybe option B) is sufficient to rename the > > > dir path /

Re: [VOTE] KIP-808: Add support for unix epoch precision in TimestampConverter SMT

2022-02-11 Thread John Roesler
Thanks, Julien! I’m +1 (binding) -John On Fri, Feb 11, 2022, at 08:42, Dongjin Lee wrote: > +1 (non-binding) > > Thanks for the KIP! > > Thanks, > Dongjin > > On Fri, Feb 11, 2022, 10:16 PM Julien Chanaud > wrote: > >> Bumping this vote. >> >> We have 2 binding votes so far. >> >> The

Re: [VOTE] KIP-769: Connect APIs to list all connector plugins and retrieve their configuration definitions

2022-02-11 Thread John Roesler
+1 (binding) Thanks, John On Fri, Feb 11, 2022, at 08:43, Dongjin Lee wrote: > +1 (non-binding) > > Thanks, > Dongjin > > On Wed, Feb 9, 2022, 2:07 AM Mickael Maison > wrote: > >> Bumping this vote. >> >> We have 1 binding and 4 non-binding votes so far. >> Let me know if you have any feedback.

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-11 Thread John Roesler
consensus around this. > Will add the deprecation notes these days and wait for any additional > feedback on this topic before wrapping up the KIP. > > > On Fri, 11 Feb 2022 at 04:03, John Roesler wrote: > >> Thanks for the update, Jorge! >> >> I just read over th

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-10 Thread John Roesler
ang Wang wrote: > > > I'm +1 on John's point 3) for punctuations. > > > > And I think if people are on the same page that a reference equality check > > per record is not a huge overhead, I think doing that enforcement is better > > than documentations and hand-wavy undef

Re: [ANNOUNCE] New committer: Luke Chen

2022-02-09 Thread John Roesler
Congratulations, Luke! -John On Wed, Feb 9, 2022, at 19:33, Mayuresh Gharat wrote: > Congratulations Luke! > > Thanks, > > Mayuresh > > On Wed, Feb 9, 2022, 5:24 PM Ismael Juma wrote: > >> Congratulations Luke! >> >> On Wed, Feb 9, 2022 at 3:22 PM Guozhang Wang wrote: >> >> > The PMC for

Re: [DISCUSS] KIP-634: Complementary support for headers in Kafka Streams DSL

2022-02-09 Thread John Roesler
Hello Jorge, Thanks for bringing this up again! I've just read over the current version of the KIP. 1) I wonder if we really need RecordValue, since we now have Record, and they are almost the same, both in API and in purpose. What do you think about instead adding topic and partition to

Re: [DISCUSS] Should we automatically close stale PRs?

2022-02-09 Thread John Roesler
Thanks for that list, Nikolay, I've just closed them all. And thanks to you all for working to keep Kafka development healthy! -John On Wed, 2022-02-09 at 14:19 +0300, Nikolay Izhikov wrote: > Hello, guys. > > I made a quick search throw oldest PRs. > Looks like the following list can be

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-09 Thread John Roesler
Thanks for the KIP Jorge, I'm in support of your proposal. 1) I do agree with Guozhang's point (1). I think the cleanest approach. I think it's cleaner and better to keep the enforcement internal to the framework than to introduce a public API or context wrapper for processors to use explicitly.

Re: [DISCUSS] KIP-816: Topology changes without local state reset

2022-02-07 Thread John Roesler
ddresses a semantic > bug: specifically that StateStores are tightly coupled to Tasks, which is > unnecessary. Reducing this coupling would add no complexity, and > potentially simplify other processes in the future. > > Regards, > > Nick > > On Sat, 5 Feb 2022 a

Re: [DISCUSS] KIP-816: Topology changes without local state reset

2022-02-05 Thread John Roesler
Hello all, Thanks for the KIP, Nick! Based on this conversation, I think I might have misread the KIP, but it looks like Nick is just proposing a small fix to the existing compatability mechanism. Although we tell people to avoid changing topologies on the fly in general, we also tell them

Re: [VOTE] KIP-591: Add Kafka Streams config to set default state store

2022-01-28 Thread John Roesler
Hi Luke, Thanks for the KIP! I'm +1 (binding) on your KIP. Regarding this last question about chaning Materialized and/or Stores, I think it might actually be best to drop that part of the proposal. The primary benefit of your proposal is in the cases when the user doesn't want to specify the

[jira] [Created] (KAFKA-13622) Revisit the complexity of position tracking in state stores

2022-01-26 Thread John Roesler (Jira)
John Roesler created KAFKA-13622: Summary: Revisit the complexity of position tracking in state stores Key: KAFKA-13622 URL: https://issues.apache.org/jira/browse/KAFKA-13622 Project: Kafka

[jira] [Resolved] (KAFKA-13608) Implement Position restoration for all in-memory state stores

2022-01-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13608. -- Resolution: Duplicate > Implement Position restoration for all in-memory state sto

[jira] [Resolved] (KAFKA-13524) IQv2: Implement KeyQuery from the RecordCache

2022-01-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13524. -- Resolution: Fixed > IQv2: Implement KeyQuery from the RecordCa

Re: [VOTE] KIP-811: Add config min.repartition.purge.interval.ms to Kafka Streams

2022-01-14 Thread John Roesler
Thanks for the KIP, Nick! +1 (binding) -John On Fri, Jan 14, 2022, at 07:40, Bruno Cadonna wrote: > Hi Nick, > > Since the title of the KIP slightly changed after the vote was opened > also the link to the KIP changed as a result. This is should be a > working link: > >

Re: [VOTE] KIP-810: Allow producing records with null values in Kafka Console Producer

2022-01-05 Thread John Roesler
Thanks Mickael! I’m +1 (binding) Thanks, John On Tue, Jan 4, 2022, at 08:53, Israel Ekpo wrote: > Thanks for the KIP, Mickael > > Looks great! > > +1 (non-binding) > > > > On Tue, Jan 4, 2022 at 5:24 AM Mickael Maison > wrote: > >> Hi, >> >> I'd like to start a vote on KIP-810 that adds the

[jira] [Resolved] (KAFKA-13525) IQv2: Implement KeyQuery from the KIP

2021-12-20 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13525. -- Resolution: Fixed > IQv2: Implement KeyQuery from the

[jira] [Created] (KAFKA-13557) IQv2: Remove swapResult from the public API

2021-12-20 Thread John Roesler (Jira)
John Roesler created KAFKA-13557: Summary: IQv2: Remove swapResult from the public API Key: KAFKA-13557 URL: https://issues.apache.org/jira/browse/KAFKA-13557 Project: Kafka Issue Type: Sub

[jira] [Created] (KAFKA-13554) Rename RangeQuery to KeyRangeQuery

2021-12-17 Thread John Roesler (Jira)
John Roesler created KAFKA-13554: Summary: Rename RangeQuery to KeyRangeQuery Key: KAFKA-13554 URL: https://issues.apache.org/jira/browse/KAFKA-13554 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-13553) Add DSL stores to IQv2StoreIntegrationTest

2021-12-17 Thread John Roesler (Jira)
John Roesler created KAFKA-13553: Summary: Add DSL stores to IQv2StoreIntegrationTest Key: KAFKA-13553 URL: https://issues.apache.org/jira/browse/KAFKA-13553 Project: Kafka Issue Type: Sub

[jira] [Resolved] (KAFKA-13492) IQ Parity: queries for key/value store range and scan

2021-12-16 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13492. -- Resolution: Fixed > IQ Parity: queries for key/value store range and s

Re: [VOTE] KIP-806: Add session and window query over KV-store in IQv2

2021-12-15 Thread John Roesler
:34 -0600, John Roesler wrote: > Thanks for the update, Patrick! > > Tl;dr: I'm +1 (binding) > > I just reviewed the KIP again (I hope you don't mind, I > fixed a couple of missed renames in the text and examples). > > One of the design of IQv2 is to make proposing and evolv

[jira] [Created] (KAFKA-13548) IQv2: revisit WindowKeyQuery and WindowRangeQuery

2021-12-15 Thread John Roesler (Jira)
John Roesler created KAFKA-13548: Summary: IQv2: revisit WindowKeyQuery and WindowRangeQuery Key: KAFKA-13548 URL: https://issues.apache.org/jira/browse/KAFKA-13548 Project: Kafka Issue Type

Re: [VOTE] KIP-806: Add session and window query over KV-store in IQv2

2021-12-15 Thread John Roesler
KIPs. > > Best, > Patrick > > On Tue, Dec 14, 2021 at 1:02 AM Guozhang Wang wrote: > > > Hi John, > > > > Please see my follow-up comments inlined below. > > > > On Mon, Dec 13, 2021 at 2:26 PM John Roesler wrote: > > > > > H

[jira] [Resolved] (KAFKA-13532) Flaky test KafkaStreamsTest.testInitializesAndDestroysMetricsReporters

2021-12-14 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13532. -- Resolution: Cannot Reproduce I looked in every branch build I could find, and didn't see any

Re: [VOTE] KIP-806: Add session and window query over KV-store in IQv2

2021-12-13 Thread John Roesler
Hi Patrick, thanks for the KIP! I hope you, Guozhang, and Luke don't mind if I share some thoughts: 1. I think you just meant to remove that private constructor from the KIP, right? 2. I think WindowRangeQuery#withWindowRage(windowLower, windowUpper) is the version that yields an iterator over

[jira] [Created] (KAFKA-13526) IQv2: Consider more generic logic for mapping between binary and typed queries

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13526: Summary: IQv2: Consider more generic logic for mapping between binary and typed queries Key: KAFKA-13526 URL: https://issues.apache.org/jira/browse/KAFKA-13526

[jira] [Created] (KAFKA-13525) IQv2: Implement KeyQuery from the KIP

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13525: Summary: IQv2: Implement KeyQuery from the KIP Key: KAFKA-13525 URL: https://issues.apache.org/jira/browse/KAFKA-13525 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-13524) IQv2: Add option to query from caches

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13524: Summary: IQv2: Add option to query from caches Key: KAFKA-13524 URL: https://issues.apache.org/jira/browse/KAFKA-13524 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-13523) Implement IQv2 support in global stores

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13523: Summary: Implement IQv2 support in global stores Key: KAFKA-13523 URL: https://issues.apache.org/jira/browse/KAFKA-13523 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-13506) IQv2: Transmit position to standbys

2021-12-08 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13506. -- Resolution: Fixed > IQv2: Transmit position to stand

[jira] [Resolved] (KAFKA-13491) Implement IQv2 Framework

2021-12-08 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13491. -- Resolution: Fixed > Implement IQv2 Framew

[jira] [Created] (KAFKA-13522) IQv2: Implement position tracking and bounding in API

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13522: Summary: IQv2: Implement position tracking and bounding in API Key: KAFKA-13522 URL: https://issues.apache.org/jira/browse/KAFKA-13522 Project: Kafka Issue

[jira] [Created] (KAFKA-13506) IQv2: Transmit position to standbys

2021-12-03 Thread John Roesler (Jira)
John Roesler created KAFKA-13506: Summary: IQv2: Transmit position to standbys Key: KAFKA-13506 URL: https://issues.apache.org/jira/browse/KAFKA-13506 Project: Kafka Issue Type: Sub-task

Re: [VOTE] KIP-805: Add range and scan query support in IQ v2

2021-12-01 Thread John Roesler
Thanks for the KIP, Vicky! I’m +1 (binding) -John On Tue, Nov 30, 2021, at 14:51, Vasiliki Papavasileiou wrote: > Hello everyone, > > I would like to start a vote for KIP-805 that adds range and scan KeyValue > queries in IQ2. > > The KIP can be found here: >

[jira] [Resolved] (KAFKA-13498) IQv2: Track Position in remaining stores

2021-12-01 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13498. -- Resolution: Fixed > IQv2: Track Position in remaining sto

[jira] [Created] (KAFKA-13498) IQv2: Track Position in remaining stores

2021-12-01 Thread John Roesler (Jira)
John Roesler created KAFKA-13498: Summary: IQv2: Track Position in remaining stores Key: KAFKA-13498 URL: https://issues.apache.org/jira/browse/KAFKA-13498 Project: Kafka Issue Type: Sub

[jira] [Created] (KAFKA-13492) IQ Parity: queries for key/value store range and scan

2021-11-30 Thread John Roesler (Jira)
John Roesler created KAFKA-13492: Summary: IQ Parity: queries for key/value store range and scan Key: KAFKA-13492 URL: https://issues.apache.org/jira/browse/KAFKA-13492 Project: Kafka Issue

[jira] [Created] (KAFKA-13491) Implement IQv2 Framework

2021-11-30 Thread John Roesler (Jira)
John Roesler created KAFKA-13491: Summary: Implement IQv2 Framework Key: KAFKA-13491 URL: https://issues.apache.org/jira/browse/KAFKA-13491 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-805: Add range and scan query support in IQ v2

2021-11-29 Thread John Roesler
Thanks for the KIP, Vicky! This KIP will help fill in the parity gap between IQ and IQv2. One thing I noticed, which looks like just a typo is that the value type of the proposed RangeQuery should probably be KeyValueIterator, right? Otherwise, it looks good to me! Thanks, -John On Mon,

Re: [VOTE] KIP-799: Align behaviour for producer callbacks with documented behaviour

2021-11-29 Thread John Roesler
Thanks, Séamus! I'm +1 (binding). On Mon, 2021-11-29 at 16:14 +, Séamus Ó Ceanainn wrote: > Hi everyone, > > I'd like to start a vote for KIP-799: Align behaviour for producer > callbacks with documented behaviour >

[jira] [Resolved] (KAFKA-13480) IQv2: Track Position in KeyValue stores

2021-11-24 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13480. -- Resolution: Fixed > IQv2: Track Position in KeyValue sto

[jira] [Created] (KAFKA-13480) IQv2: Track Position in KeyValue stores

2021-11-24 Thread John Roesler (Jira)
John Roesler created KAFKA-13480: Summary: IQv2: Track Position in KeyValue stores Key: KAFKA-13480 URL: https://issues.apache.org/jira/browse/KAFKA-13480 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-13479) Interactive Query v2

2021-11-24 Thread John Roesler (Jira)
John Roesler created KAFKA-13479: Summary: Interactive Query v2 Key: KAFKA-13479 URL: https://issues.apache.org/jira/browse/KAFKA-13479 Project: Kafka Issue Type: New Feature

Re: [VOTE} KIP-796: Interactive Query v2

2021-11-22 Thread John Roesler
gt; > > (non-binding) from me. > > > > > > Thank you John! > > > > > > On Thu, Nov 18, 2021 at 9:45 PM Patrick Stuedi > > > > > wrote: > > > > > > > +1 (non-binding), thanks John! > > > > -Patrick > > >

Re: [VOTE] KIP-798 Add possibility to write kafka headers in Kafka Console Producer

2021-11-20 Thread John Roesler
Hi Florin, Thanks for the KIP! I think the assumption that header values are UTF-8 strings might not hold up in the long run, but it seems like we can easily add a property later to specify the format. It seems like this scope is probably a handy addition on its own. I’m +1 (binding)

Re: [ANNOUNCE] New Kafka PMC Member: Tom Bentley

2021-11-18 Thread John Roesler
Congratulations, Tom! On Thu, Nov 18, 2021, at 17:53, Konstantine Karantasis wrote: > Congratulations Tom! > > Konstantine > > > On Thu, Nov 18, 2021 at 2:44 PM Luke Chen wrote: > >> Congrats, Tom! >> >> Guozhang Wang 於 2021年11月19日 週五 上午1:13 寫道: >> >> > Congrats Tom! >> > >> > Guozhang >> > >>

Re: Errors thrown from a KStream transformer are swallowed, eg. StackOverflowError

2021-11-18 Thread John Roesler
Thanks for pointing that out, Scott! You’re totally right; that should be a Throwable. Just to put it out there, do you want to just send a quick PR? If not, no worries. I’m just asking because it seems like you’ve already done the hard part and it might be nice to get the contribution credit.

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
seems > more exception types are possible from KIP-216: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-216%3A+IQ+should+throw+different+exceptions+for+different+errors, > should we include all in the javadocs? > > > Guozhang > > > > On Wed, Nov 17, 2021

Re: [VOTE] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
Obviously, I will be voting +1 (binding) Thanks, -John On Wed, 2021-11-17 at 17:27 -0600, John Roesler wrote: > Hello all, > > I'd like to open the vote for KIP-796, which proposes > a revamp of the Interactive Query APIs in Kafka Streams. > > The proposal is here: > htt

[VOTE} KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
Hello all, I'd like to open the vote for KIP-796, which proposes a revamp of the Interactive Query APIs in Kafka Streams. The proposal is here: https://cwiki.apache.org/confluence/x/34xnCw Thanks to all who reviewed the proposal, and thanks in advance for taking the time to vote! Thank you,

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
Wang wrote: > Thanks John! Some more thoughts inlined below. > > On Mon, Nov 15, 2021 at 10:07 PM John Roesler wrote: > > > Thanks for the review, Guozhang! > > > > 1. This is a great point. I fell into the age-old trap of > > only considering the sim

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
xplicitly to the KIP and > also to Javadocs if possible? That would guide the future contributors. > WDYT? > > The other question I have is (may be irrelevant) but with these changes, is > there going to be any impact on remote state store querying capabilities? > > Thanks! > S

[jira] [Resolved] (KAFKA-13426) Add recordMetadata to StateStoreContext

2021-11-16 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13426. -- Resolution: Fixed > Add recordMetadata to StateStoreCont

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-15 Thread John Roesler
nd up with a single function in > `KafkaStreams`, and the inner store always only need to implement the raw > query types. Of course doing this would not be so easy given the fact > described in 1) above, but I feel this would be a good way to first > abstract away this tech debt, and

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-15 Thread John Roesler
in InteractiveQueryRequest and InteractiveQueryResult > > is named differently, any particular reason? > > > > Best, > > Patrick > > > > > > On Fri, Nov 12, 2021 at 12:29 AM John Roesler wrote: > > > > > Thanks for taking a look, Sophie! > > >

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-11 Thread John Roesler
refer to Optional. > Not > sure which is intended for this API, but if is supposed to be the return > type, do you perhaps > mean for it to be Optional.ofEmpty() and Optional.of(non-empty set) > rather than Optional.of(empty set) and Optional.of(non-empty set) ? > > On Thu, Nov 11

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-11 Thread John Roesler
Hello again, all, Just bumping this discussion on a new, more flexible Interactive Query API in Kafka Streams. If there are no concerns, I'll go ahead and call a vote on Monday. Thanks! -John On Tue, 2021-11-09 at 17:37 -0600, John Roesler wrote: > Hello all, > > I'd like

Re: [VOTE] KIP-779: Allow Source Tasks to Handle Producer Exceptions

2021-11-11 Thread John Roesler
Thanks, Knowles! I'm +1 (binding) -John On Wed, 2021-11-10 at 12:42 -0500, Christopher Shannon wrote: > +1 (non-binding). This looks good to me and will be useful as a way to > handle producer errors. > > On Mon, Nov 8, 2021 at 8:55 AM Knowles Atchison Jr > wrote: > > > Good morning, > > >

Re: [DISCUSS] KIP-799 Align behaviour for producer callbacks with documented behaviour

2021-11-11 Thread John Roesler
that > refactoring done, so any discussion on that topic can take place in Github > / JIRA. > > Regards, > Séamus. > > On Thu, 11 Nov 2021 at 14:33, John Roesler wrote: > > > Thanks for the KIP, Séamus! > > > > I agree that the current situation you’re des

Re: [DISCUSS] KIP-799 Align behaviour for producer callbacks with documented behaviour

2021-11-11 Thread John Roesler
Thanks for the KIP, Séamus! I agree that the current situation you’re describing doesn’t seem ideal, and it’s probably worth a slight behavior change to fix it. It’s too bad that we introduced that placeholder record, since it seems less error prone for users if we have the invariant that

[jira] [Resolved] (KAFKA-13429) Update gitignore to include new modules

2021-11-10 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13429. -- Resolution: Fixed > Update gitignore to include new modu

<    1   2   3   4   5   6   7   8   9   10   >