[DISCUSS] Please review 2.8.0 blog post

2021-04-01 Thread John Roesler
Hello all, In the steady march toward the Apache Kafka 2.8.0 release, I have prepared a draft of the release announcement post: https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache5 If you have a moment, I would greatly appreciate your reviews. Thank you, -John

Re: [DISCUSS] KIP-718: Make KTable Join on Foreign key unopinionated

2021-04-01 Thread John Roesler
ang, > > Apologies for the late answer. Originally that was my proposal - to > piggyback on the provided materialisation method ( > https://issues.apache.org/jira/browse/KAFKA-10383). > John Roesler suggested to us to provide even further fine tuning on API > level parameter

Re: [DISCUSS] KIP-633: Drop 24 hour default of grace period in Streams

2021-03-31 Thread John Roesler
Thanks for bringing this up, Sophie! This has indeed been a pain point for a lot of people. It's a really thorny issue with no obvious "right" solution. I think your proposal is a good one. Thanks, -John On Wed, 2021-03-31 at 13:28 -0700, Sophie Blee-Goldman wrote: > Hey all, > > It's finally

Re: [VOTE] 2.6.2 RC0

2021-03-31 Thread John Roesler
Hi Justin, Thanks for bringing this up and for your advice in resolving it. I've carved out the specific chunk of work relating to copyright, and wanted to call your attention to a specific proposal I have to fix this situation:

Re: [VOTE] 2.8.0 RC0

2021-04-06 Thread John Roesler
Hello again, all, I am closing this vote in favor of the 2.8.0 RC1 thread. Thank you, John On Tue, 2021-03-30 at 17:11 -0500, John Roesler wrote: > Hello again, all, > > I just wanted to mention that I am aware of Justin's > concerns in the 2.6.2 thread: > https://lists.apache.

Subject: [VOTE] 2.8.0 RC1

2021-04-06 Thread John Roesler
Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 2.8.0. This is a major release that includes many new features, including: * Early-access release of replacing Zookeeper with a self- managed quorum * Add Describe Cluster API * Support

Re: [VOTE] KIP-725: Streamlining configurations for TimeWindowedDeserializer.

2021-04-05 Thread John Roesler
Thanks, Sagar! I’m +1 (binding) -John On Mon, Apr 5, 2021, at 21:35, Sophie Blee-Goldman wrote: > Thanks for the KIP! +1 (binding) from me > > Cheers, > Sophie > > On Mon, Apr 5, 2021 at 7:13 PM Sagar wrote: > > > Hi All, > > > > I would like to start voting on the following KIP: > > > >

Re: [DISCUSS] Please review 2.8.0 blog post

2021-04-05 Thread John Roesler
t; > On Mon, Apr 5, 2021 at 6:29 PM Adam Bellemare > wrote: > > > Read it all. It looks good to me in terms of structure and content. I am > > not sufficiently up to date on all the features that are otherwise included > > in 2.8.0, but the ones listed seem very prominent!

Re: [VOTE] KIP-633: Drop 24 hour default of grace period in Streams

2021-04-05 Thread John Roesler
Thanks, Sophie, I’m +1 (binding) -John On Mon, Apr 5, 2021, at 21:34, Sophie Blee-Goldman wrote: > Hey all, > > I'd like to start the voting on KIP-633, to drop the awkward 24 hour grace > period and improve the API to raise visibility on an important concept in > Kafka Streams: grace period

Re: [VOTE] 2.8.0 RC0

2021-03-30 Thread John Roesler
release, but wanted to get RC0 out asap for testing. Thank you, John On Tue, 2021-03-30 at 16:37 -0500, John Roesler wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of Apache Kafka > 2.8.0. This is a major release that inclu

[VOTE] 2.8.0 RC0

2021-03-30 Thread John Roesler
Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 2.8.0. This is a major release that includes many new features, including: * Early-access release of replacing Zookeeper with a self- managed quorum * Add Describe Cluster API * Support

Re: [DISCUSS] KIP-718: Make KTable Join on Foreign key unopinionated

2021-04-06 Thread John Roesler
d MaterializedSubscription since it seems we want to allow > > users to fully customize this store? > > > > Guozhang > > > > On Thu, Apr 1, 2021 at 5:28 PM John Roesler wrote: > > > > > Thanks Marco, > > > > > > Sorry if I cause

Re: [VOTE] KIP-708: Rack awareness for Kafka Streams

2021-03-11 Thread John Roesler
Thanks for the KIP! I'm +1 (binding) -John On Wed, 2021-03-10 at 13:13 +0200, Levani Kokhreidze wrote: > Hello all, > > I’d like to start the voting on KIP-708 [1] > > Best, > Levani > > [1] - > https://cwiki.apache.org/confluence/display/KAFKA/KIP-708%3A+Rack+awareness+for+Kafka+Streams >

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-03-02 Thread John Roesler
nk. > > - Dhruvil > > On Mon, Feb 22, 2021 at 7:50 AM John Roesler wrote: > > > Thanks for the heads-up, Chia-Ping, > > > > I agree it would be good to include that fix. > > > > Thanks, > > John > > > > On Mon, 2021-02-22 at 09

Re: [VOTE] KIP-725: Streamlining configurations for TimeWindowedDeserializer.

2021-04-08 Thread John Roesler
ar, +1 non-binding. Thanks again for doing this. > > > > > > Leah > > > > > > On Mon, Apr 5, 2021 at 9:40 PM John Roesler wrote: > > > > > > > Thanks, Sagar! > > > > > > > > I’m +1 (binding) > > > > > >

Re: [DISCUSS] Please review 2.8.0 blog post

2021-04-07 Thread John Roesler
the explanation John! > > > > Thanks! > > Sagar. > > > > On Tue, Apr 6, 2021 at 8:16 AM John Roesler wrote: > > > > > Oh, my apologies, Sagar, > > > > > > That link will not resolve until the release. The release > >

Re: [kafka-clients] Subject: [VOTE] 2.8.0 RC1

2021-04-07 Thread John Roesler
021 instead. > > Could we assume that was just a carry over from the previous solicitation > for 2.8.0 RC0 and you actually meant to say 2021-04-13? > > When you have a moment, please clarify. > > I am running my tests shortly and will share my results by the end of the > week. &g

Re: [VOTE] KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2

2021-04-15 Thread John Roesler
Thanks, Sophie, I'm +1 (binding) -John On Wed, 2021-04-14 at 16:59 -0700, Sophie Blee-Goldman wrote: > Hey all, > > I'd like to kick off the vote on KIP-732, to deprecate eos-alpha in Kafka > Streams and migrate away from the "eos-beta" terminology by replacing it > with "eos-v2" to shore up

[RESULTS] [VOTE] Release Kafka version 2.8.0

2021-04-18 Thread John Roesler
This vote passes with 6 +1 votes (4 binding) and no 0 or -1 votes. +1 votes PMC Members: * Bill Bejeck * Randall Hauch * Colin McCabe * John Roesler Community: * Israel Ekpo * Satish Duggana 0 votes * No votes -1 votes * No votes Vote thread: https://lists.apache.org/thread.html

[ANNOUNCE] Apache Kafka 2.8.0

2021-04-19 Thread John Roesler
, Ismael Juma, Ivan Ponomarev, Ivan Yurchenko, jackyoh, James Cheng, James Yuzawa, Jason Gustafson, Jesse Gorzinski, Jim Galasyn, John Roesler, Jorge Esteban Quilcate Otoya, José Armando García Sancio, Julien Chanaud, Julien Jean Paul Sirocchi, Justine Olshan, Kengo Seki, Kowshik Prakasam, leah, Lee

Re: [DISCUSS] KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2

2021-04-14 Thread John Roesler
Thanks for the KIP, Sophie. I think this is a great approach, and it makes perfect sense in the 3.0 release as well. -John On Wed, Apr 14, 2021, at 01:17, Bruno Cadonna wrote: > Hi Sophie, > > the KIP makes sense to me! Thank you for writing it! > > Best, > Bruno > > On 14.04.21 04:52,

Re: [VOTE] KIP-718: Make KTable Join on Foreign key unopinionated

2021-04-14 Thread John Roesler
I’m +1 (binding) Thanks, Marco! -John On Wed, Apr 14, 2021, at 07:59, Marco Aurélio Lotz wrote: > Hi all, > > I would like to start a vote on KIP-718, which proposes to make KTable join > on foreign key unopinionated in terms of persistence method (currently it > forces RocksDB usage for

Re: [kafka-clients] Re: Subject: [VOTE] 2.8.0 RC1

2021-04-14 Thread John Roesler
n, Apr 12, 2021 at 8:47 PM John Roesler > wrote: > > Good catch, Israel! > > > > I’ll make sure that gets fixed. > > > > Thanks, > > John > > > > On Mon, Apr 12, 2021, at 19:30, Israel Ekpo wrote: > > > I just noticed that with the la

Re: [DISCUSS] KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2

2021-04-14 Thread John Roesler
Hello again, Sophie, I've just had a late-breaking thought about your KIP. It might be worthwhile also printing a WARN log when the deprecated configs are used. People may not always be referencing the StreamsConfig fields in their source code when they set these configs, so it would be good to

[VOTE] 2.8.0 RC2

2021-04-14 Thread John Roesler
Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 2.8.0.This is a major release that includes many new features, including: * Early-access release of replacing Zookeeper with a self-managed quorum * Add Describe Cluster API *

Re: I have a question about apache kafka.

2021-04-08 Thread John Roesler
Hello Yun Han Nam, The users@ list is more typical for this kind of topic, but this list is fine, too. What’s the question? Thanks, John On Thu, Apr 8, 2021, at 06:25, 남윤한[Yun Han Nam] wrote: > Hi. > > I want to ask you a question about Apache Kafka, is this the right one? > > It is a

Re: [VOTE] 2.6.2 RC1

2021-04-14 Thread John Roesler
I verified the signatures and checksums, ran the tests, and spot-checked the license. I'm +1 (binding). Thanks for driving this, Sophie! -John On Thu, 2021-04-08 at 16:19 -0700, Guozhang Wang wrote: > Looked over the javadocs and web docs again. Download the jars and check > the license files

Re: [VOTE] 2.7.1 RC2

2021-04-14 Thread John Roesler
Hi Mickael, I verified the signatures and checksums, ran the tests, and spot-checked the license. I'm +1 (binding). Thanks for driving this release! -John On Fri, 2021-04-09 at 11:07 +0100, Mickael Maison wrote: > Hi, > > Here is a successful build of 2.7.1 RC2: >

Re: Subject: [VOTE] 2.8.0 RC1

2021-04-12 Thread John Roesler
cs haven't been > installed to https://kafka.apache.org/28/documentation.html yet. > > Thanks again! > > -Bill > > On Tue, Apr 6, 2021 at 5:37 PM John Roesler wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is t

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-18 Thread John Roesler
you have forgotten to update >the docs for some features in 2.8.0. Once we have a green build and passing system tests, I will cut the first RC. Thank you, John On Sun, Feb 7, 2021, at 09:59, John Roesler wrote: > Hello all, > > I have just cut the branch for 2.8 and sent the

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-16 Thread John Roesler
s > > we may add in the future > > > > Just my 2 cents. But if the pollOptions proposal would really add so much > > additional work that it would cause the 2.8 release to be significantly > > delayed, > > then that's worth taking into account as well. > > >

Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-05 Thread John Roesler
urn_on_response > > > > This idea LGTM. It not only makes minimum changes to current behavior but > > also works for KIP-695. > > > > On 2021/02/04 16:07:11, John Roesler wrote: > > > Hi Matthias, Chia-Ping, and Tom, > > > > > > Than

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-21 Thread John Roesler
; > The PRs for these have already been approved and merged to the `trunk` > branch (and backported to a few older branches). I'd like to backport these > to the `2.8` branch. > > Thanks, and best regards! > > Randall > > On Thu, Feb 18, 2021 at 10:23 AM John Roesler wr

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
Hello all, Since there haven't been any big objections to my proposed design fix, I'm updating the KIP now and opening the PR for reviews. Hopefully, we can get this merged quickly and unblock the system tests. Thanks, John On Mon, 2021-02-22 at 09:49 -0600, John Roesler wrote: > Tha

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-22 Thread John Roesler
ual at > > This error obstructs us from running integration tests so I'd like to push it > to 2.8 branch after it gets approved. > > Best Regards, > Chia-Ping > > On 2021/02/18 16:23:13, "John Roesler" wrote: > > Hello again, all. > > > > Thi

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
m to carry more information without adding more Public APIs to > Consumer. > > On 2021/02/17 05:21:06, John Roesler wrote: > > Hello again, all, > > > > Thank you for your feedback and patience. I am hopeful that > > I have been able to come up with a solution that will

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
ormation the > consumer has access to when it receives the fetch response. > > Thanks again. > > Bill > > On Mon, Feb 22, 2021 at 10:51 AM John Roesler wrote: > > > Hello all, > > > > Since there haven't been any big objections to my proposed > >

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
On Mon, 2021-02-22 at 10:25 -0600, John Roesler wrote: > Thanks, Bill, > > I was waiting for feedback on the "currentLag" proposal > before updating the KIP. Since there haven't been any > objections to my new proposal, I'm in the process of > updating the KIP document r

Re: [VOTE] KIP-766: fetch/findSessions queries with open endpoints for SessionStore/WindowStore

2021-08-17 Thread John Roesler
Thanks, Luke! I'm sorry I missed your discussion thread. The KIP looks good to me! I'm +1 (binding) Thanks, -John On Tue, 2021-08-17 at 16:40 +0800, Luke Chen wrote: > Hi all, > I'd like to start to vote for > *KIP-766: fetch/findSessions queries with open endpoints for >

Re: [DISCUSS] Apache Kafka 2.8.1 release

2021-08-16 Thread John Roesler
Thanks, David! +1 from me as well. -John On Mon, 2021-08-16 at 12:00 -0400, Bill Bejeck wrote: > Thanks for volunteering to run the release David. > > It's a +1 for me. > > Bill > > On Mon, Aug 16, 2021 at 1:14 AM Konstantine Karantasis < > kkaranta...@apache.org> wrote: > > > Thanks for

Re: [DISCUSS] KIP-655: Windowed "Distinct" Operation for KStream

2021-07-28 Thread John Roesler
t behind it. > > John, good catch about the missing overloads! > BTW, the overload with Named should be there regardless of stateful or > stateless. > > Best, > Bruno > > On 22.07.21 20:58, John Roesler wrote: > > Hi Ivan, > > > > Thanks for th

Re: [VOTE] Add TaskId field to StreamsException

2021-10-18 Thread John Roesler
Thanks, Sophie, +1 from me. I was a little apprehensive about the potential to break exception handler logic, but I agree with your comment in the "compatibility" section: Identifying the causes of exceptions is always kind of chaotic and this seems like more of an improvement than anything.

Re: KIP Process Needs Improvement

2021-10-19 Thread John Roesler
Good morning Knowles, Thank you for sharing this feedback. I think your frustration is well founded. I think most/all of the active committers carry around a sense of guilt about KIPs that we haven’t been able to review. We do have a responsibility to hold auditable, public design discussions

Re: [VOTE] KIP-775: Custom partitioners in foreign key joins

2021-09-28 Thread John Roesler
+1 (binding) Thanks, Victoria! -John On Tue, Sep 28, 2021, at 16:29, Adam Bellemare wrote: > +1 (non-binding) > > Glad to see this in here :) > > On Tue, Sep 28, 2021 at 5:11 PM Bill Bejeck wrote: > >> +1 (binding) >> >> On Tue, Sep 28, 2021 at 12:59 PM Matthias J. Sax wrote: >> >> > +1

Re: [DISCUSS] KIP-775: Custom partitioners in foreign key joins

2021-09-22 Thread John Roesler
Thanks for the KIP, Victoria! This is a great catch. As fas as my memory serves, we totally missed this case in the design of FK joins. tl;dr: I'm wondering if it might be better to instead introduce a way to specify the partitioning scheme when we create a KTable and then just use it when we

Re: [DISCUSS] KIP-775: Custom partitioners in foreign key joins

2021-09-22 Thread John Roesler
I'm slightly leaning towards the > current proposal in the KIP doc, i.e. just fixing for this FK operator > only, with the easy approach that requires users themselves to set > partitioners accordingly. > > > > Guozhang > > > > On Wed, Sep 22, 2021 at 10:29 AM Joh

Re: [DISCUSS] KIP-775: Custom partitioners in foreign key joins

2021-09-22 Thread John Roesler
` builder methods. If that proposal isn't clear, I'd be happy to provide a code sketch. Thanks in advance for your consideration, and thanks again for driving this, -John On Wed, 2021-09-22 at 17:34 -0500, John Roesler wrote: > Thanks, Guozhang! > > I agree it would be bigger in scope to

Re: [VOTE] KIP-810: Allow producing records with null values in Kafka Console Producer

2022-01-05 Thread John Roesler
Thanks Mickael! I’m +1 (binding) Thanks, John On Tue, Jan 4, 2022, at 08:53, Israel Ekpo wrote: > Thanks for the KIP, Mickael > > Looks great! > > +1 (non-binding) > > > > On Tue, Jan 4, 2022 at 5:24 AM Mickael Maison > wrote: > >> Hi, >> >> I'd like to start a vote on KIP-810 that adds the

Re: [VOTE} KIP-796: Interactive Query v2

2021-11-22 Thread John Roesler
gt; > > (non-binding) from me. > > > > > > Thank you John! > > > > > > On Thu, Nov 18, 2021 at 9:45 PM Patrick Stuedi > > > > > wrote: > > > > > > > +1 (non-binding), thanks John! > > > > -Patrick > > >

Re: [VOTE] KIP-798 Add possibility to write kafka headers in Kafka Console Producer

2021-11-20 Thread John Roesler
Hi Florin, Thanks for the KIP! I think the assumption that header values are UTF-8 strings might not hold up in the long run, but it seems like we can easily add a property later to specify the format. It seems like this scope is probably a handy addition on its own. I’m +1 (binding)

Re: [ANNOUNCE] New Kafka PMC Member: Tom Bentley

2021-11-18 Thread John Roesler
Congratulations, Tom! On Thu, Nov 18, 2021, at 17:53, Konstantine Karantasis wrote: > Congratulations Tom! > > Konstantine > > > On Thu, Nov 18, 2021 at 2:44 PM Luke Chen wrote: > >> Congrats, Tom! >> >> Guozhang Wang 於 2021年11月19日 週五 上午1:13 寫道: >> >> > Congrats Tom! >> > >> > Guozhang >> > >>

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
xplicitly to the KIP and > also to Javadocs if possible? That would guide the future contributors. > WDYT? > > The other question I have is (may be irrelevant) but with these changes, is > there going to be any impact on remote state store querying capabilities? > > Thanks! > S

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
Wang wrote: > Thanks John! Some more thoughts inlined below. > > On Mon, Nov 15, 2021 at 10:07 PM John Roesler wrote: > > > Thanks for the review, Guozhang! > > > > 1. This is a great point. I fell into the age-old trap of > > only considering the sim

Re: [VOTE] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
Obviously, I will be voting +1 (binding) Thanks, -John On Wed, 2021-11-17 at 17:27 -0600, John Roesler wrote: > Hello all, > > I'd like to open the vote for KIP-796, which proposes > a revamp of the Interactive Query APIs in Kafka Streams. > > The proposal is here: > htt

[VOTE} KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
Hello all, I'd like to open the vote for KIP-796, which proposes a revamp of the Interactive Query APIs in Kafka Streams. The proposal is here: https://cwiki.apache.org/confluence/x/34xnCw Thanks to all who reviewed the proposal, and thanks in advance for taking the time to vote! Thank you,

Re: [DISCUSS] KIP-799 Align behaviour for producer callbacks with documented behaviour

2021-11-11 Thread John Roesler
Thanks for the KIP, Séamus! I agree that the current situation you’re describing doesn’t seem ideal, and it’s probably worth a slight behavior change to fix it. It’s too bad that we introduced that placeholder record, since it seems less error prone for users if we have the invariant that

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-11 Thread John Roesler
Hello again, all, Just bumping this discussion on a new, more flexible Interactive Query API in Kafka Streams. If there are no concerns, I'll go ahead and call a vote on Monday. Thanks! -John On Tue, 2021-11-09 at 17:37 -0600, John Roesler wrote: > Hello all, > > I'd like

Re: [VOTE] KIP-779: Allow Source Tasks to Handle Producer Exceptions

2021-11-11 Thread John Roesler
Thanks, Knowles! I'm +1 (binding) -John On Wed, 2021-11-10 at 12:42 -0500, Christopher Shannon wrote: > +1 (non-binding). This looks good to me and will be useful as a way to > handle producer errors. > > On Mon, Nov 8, 2021 at 8:55 AM Knowles Atchison Jr > wrote: > > > Good morning, > > >

Re: [DISCUSS] KIP-799 Align behaviour for producer callbacks with documented behaviour

2021-11-11 Thread John Roesler
that > refactoring done, so any discussion on that topic can take place in Github > / JIRA. > > Regards, > Séamus. > > On Thu, 11 Nov 2021 at 14:33, John Roesler wrote: > > > Thanks for the KIP, Séamus! > > > > I agree that the current situation you’re des

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-11 Thread John Roesler
refer to Optional. > Not > sure which is intended for this API, but if is supposed to be the return > type, do you perhaps > mean for it to be Optional.ofEmpty() and Optional.of(non-empty set) > rather than Optional.of(empty set) and Optional.of(non-empty set) ? > > On Thu, Nov 11

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-15 Thread John Roesler
in InteractiveQueryRequest and InteractiveQueryResult > > is named differently, any particular reason? > > > > Best, > > Patrick > > > > > > On Fri, Nov 12, 2021 at 12:29 AM John Roesler wrote: > > > > > Thanks for taking a look, Sophie! > > >

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-15 Thread John Roesler
nd up with a single function in > `KafkaStreams`, and the inner store always only need to implement the raw > query types. Of course doing this would not be so easy given the fact > described in 1) above, but I feel this would be a good way to first > abstract away this tech debt, and

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-11-17 Thread John Roesler
seems > more exception types are possible from KIP-216: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-216%3A+IQ+should+throw+different+exceptions+for+different+errors, > should we include all in the javadocs? > > > Guozhang > > > > On Wed, Nov 17, 2021

Re: Errors thrown from a KStream transformer are swallowed, eg. StackOverflowError

2021-11-18 Thread John Roesler
Thanks for pointing that out, Scott! You’re totally right; that should be a Throwable. Just to put it out there, do you want to just send a quick PR? If not, no worries. I’m just asking because it seems like you’ve already done the hard part and it might be nice to get the contribution credit.

Re: [DISCUSS] KIP-779: Allow Source Tasks to Handle Producer Exceptions

2021-10-27 Thread John Roesler
Good morning, Knowles, Thanks for the KIP! To address your latest questions, it is fine to call for a vote if a KIP doesn't generate much discussion. Either the KIP was just not controversial enough for anyone to comment, in which case a vote is appropriate; or no one had time to review it, in

Re: [DISCUSS] KIP-779: Allow Source Tasks to Handle Producer Exceptions

2021-10-27 Thread John Roesler
y have at the time to inform decisions on > what to do with these producer exceptions. > > I have updated the KIP and PR. > > Knowles > > On Wed, Oct 27, 2021 at 1:03 PM John Roesler wrote: > > > Good morning, Knowles, > > > > Thanks for the KIP! > >

Re: [VOTE] KIP-791: Add Record Metadata to State Store Context

2021-11-09 Thread John Roesler
+1 (binding) from me. Thanks, Patrick! On Mon, 2021-11-08 at 14:08 -0800, Guozhang Wang wrote: > +1, thanks Patrick! > > > Guozhang > > On Mon, Nov 8, 2021 at 5:44 AM Vasiliki Papavasileiou > wrote: > > > Hi Patrick, > > > > Having the recordMetadata available in the state stores is

[DISCUSS] KIP-796: Interactive Query v2

2021-11-09 Thread John Roesler
Hello all, I'd like to start the discussion for KIP-796, which proposes a revamp of the Interactive Query APIs in Kafka Streams. The proposal is here: https://cwiki.apache.org/confluence/x/34xnCw I look forward to your feedback! Thank you, -John

Re: [DISCUSS] KIP-791: Add Record Metadata to State Store Context

2021-11-05 Thread John Roesler
Thanks for the KIP, Patrick! It looks like you addressed Guozhang's and Bruno's very good feeback, and I like the result. The example especially helps clarify how this property might be useful. I'm in favor of this proposal. Thanks, -John On Fri, 2021-11-05 at 12:03 +0100, Bruno Cadonna wrote:

Re: [DISCUSS] KIP-805: Add range and scan query support in IQ v2

2021-11-29 Thread John Roesler
Thanks for the KIP, Vicky! This KIP will help fill in the parity gap between IQ and IQv2. One thing I noticed, which looks like just a typo is that the value type of the proposed RangeQuery should probably be KeyValueIterator, right? Otherwise, it looks good to me! Thanks, -John On Mon,

Re: [VOTE] KIP-799: Align behaviour for producer callbacks with documented behaviour

2021-11-29 Thread John Roesler
Thanks, Séamus! I'm +1 (binding). On Mon, 2021-11-29 at 16:14 +, Séamus Ó Ceanainn wrote: > Hi everyone, > > I'd like to start a vote for KIP-799: Align behaviour for producer > callbacks with documented behaviour >

Re: [VOTE] KIP-805: Add range and scan query support in IQ v2

2021-12-01 Thread John Roesler
Thanks for the KIP, Vicky! I’m +1 (binding) -John On Tue, Nov 30, 2021, at 14:51, Vasiliki Papavasileiou wrote: > Hello everyone, > > I would like to start a vote for KIP-805 that adds range and scan KeyValue > queries in IQ2. > > The KIP can be found here: >

Re: [VOTE] KIP-806: Add session and window query over KV-store in IQv2

2021-12-15 Thread John Roesler
:34 -0600, John Roesler wrote: > Thanks for the update, Patrick! > > Tl;dr: I'm +1 (binding) > > I just reviewed the KIP again (I hope you don't mind, I > fixed a couple of missed renames in the text and examples). > > One of the design of IQv2 is to make proposing and evolv

Re: [VOTE] KIP-806: Add session and window query over KV-store in IQv2

2021-12-15 Thread John Roesler
KIPs. > > Best, > Patrick > > On Tue, Dec 14, 2021 at 1:02 AM Guozhang Wang wrote: > > > Hi John, > > > > Please see my follow-up comments inlined below. > > > > On Mon, Dec 13, 2021 at 2:26 PM John Roesler wrote: > > > > > H

Re: [VOTE] KIP-806: Add session and window query over KV-store in IQv2

2021-12-13 Thread John Roesler
Hi Patrick, thanks for the KIP! I hope you, Guozhang, and Luke don't mind if I share some thoughts: 1. I think you just meant to remove that private constructor from the KIP, right? 2. I think WindowRangeQuery#withWindowRage(windowLower, windowUpper) is the version that yields an iterator over

Re: [DISCUSS] KIP-763: Range queries with open endpoints

2021-07-20 Thread John Roesler
Thanks for the KIP, Patrick! I think your KIP is well motivated. It's definitely a bummer to have to iterate over the full store as a workaround for open-ended ranges. I agree with your pragmatic approach here. We have recently had a couple of other contributions to the store APIs (prefix and

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-22 Thread John Roesler
Hello Konstantine, Someone just called to my attention that KAFKA-12724 had not been marked as a 3.0 blocker. We never added 2.8 to the Streams upgrade system test suite. This isn't a blocker in that it is a problem, but we should make sure that Streams is actually upgradable before releasing

Re: [DISCUSS] KIP-655: Windowed "Distinct" Operation for KStream

2021-07-22 Thread John Roesler
) distinct(Materialized) It's a small update, but an important one, since people will inevitably need to customize the state store for the operation. Thanks, John On Thu, 2021-07-22 at 13:58 -0500, John Roesler wrote: > Hi Ivan, > > Thanks for the reply. > > 1. I think I might hav

Re: [VOTE] KIP-763: Range queries with open endpoints

2021-07-22 Thread John Roesler
Thank you, Patrick, +1 (binding) from me as well! Thanks, -John On Thu, 2021-07-22 at 10:40 +0200, Bruno Cadonna wrote: > Hi Patrick, > > Thank you for the KIP! > > +1 (binding) > > Best, > Bruno > > On 22.07.21 03:47, Luke Chen wrote: > > Hi Patrick, > > I like this KIP! > > > > +1

Re: [DISCUSS] KIP-655: Windowed "Distinct" Operation for KStream

2021-07-22 Thread John Roesler
teach KStreams, we always say that > KStreams are just like other streaming APIs, and they have roots in SQL > queries. Users know what `distinct` is and they expect it to be in the API. > > > Regards, > > Ivan > > > 13.07.2021 0:10, John Roesler пишет: > >

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-11 Thread John Roesler
consensus around this. > Will add the deprecation notes these days and wait for any additional > feedback on this topic before wrapping up the KIP. > > > On Fri, 11 Feb 2022 at 04:03, John Roesler wrote: > >> Thanks for the update, Jorge! >> >> I just read over th

Re: [VOTE] KIP-808: Add support for unix epoch precision in TimestampConverter SMT

2022-02-11 Thread John Roesler
Thanks, Julien! I’m +1 (binding) -John On Fri, Feb 11, 2022, at 08:42, Dongjin Lee wrote: > +1 (non-binding) > > Thanks for the KIP! > > Thanks, > Dongjin > > On Fri, Feb 11, 2022, 10:16 PM Julien Chanaud > wrote: > >> Bumping this vote. >> >> We have 2 binding votes so far. >> >> The

Re: [VOTE] KIP-769: Connect APIs to list all connector plugins and retrieve their configuration definitions

2022-02-11 Thread John Roesler
+1 (binding) Thanks, John On Fri, Feb 11, 2022, at 08:43, Dongjin Lee wrote: > +1 (non-binding) > > Thanks, > Dongjin > > On Wed, Feb 9, 2022, 2:07 AM Mickael Maison > wrote: > >> Bumping this vote. >> >> We have 1 binding and 4 non-binding votes so far. >> Let me know if you have any feedback.

Re: [VOTE] KIP-820: Extend KStream process with new Processor API

2022-03-15 Thread John Roesler
hey also include `final String... stateStoreNames`. > And in javadocs explains that if users want to connect state stores to this > processor, they could use the `connectState` API instead. > > Otherwise, I'm +1. > > Guozhang > > On Tue, Feb 15, 2022 at 11:54 AM John Roe

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-03-06 Thread John Roesler
belatedly was that if we do set it to > > > > > > Void, then users will actually have to override the key when > > > > > > forwarding, like `record.withKey(null)`, whereas if we keep > > > > > > it is K, all users have to do is not touch the key at all. &

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-25 Thread John Roesler
at 11:07 -0600, John Roesler wrote: > Hello all, > > I'll chime in again in the interest of trying to do a better > job of keeping KIPs moving forward... > > Matthias raised some very good questions about whether the > change is really source compatible. I just checked ou

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-25 Thread John Roesler
check could be sufficient; as for the > > purpose of 2), as for this KIP itself I think it is similar to what we have > > (i.e. just base on the function name "processValue" itself) and hence are > > not sacrificed either. I do not know if > > `KStream#processValu

Re: [VOTE] KIP-812: Introduce another form of the `KafkaStreams.close()` API that forces the member to leave the consumer group

2022-02-17 Thread John Roesler
Thanks for the KIP, Sueng-Chan! I’m +1 (binding) -John On Thu, Feb 17, 2022, at 12:58, Guozhang Wang wrote: > Thanks Sueng-chan, > > I'm +1 on the proposal. > > > Guozhang > > On Tue, Feb 15, 2022 at 7:06 AM Seung-chan Ahn > wrote: > >> Hi team, >> >> I feel like we have a finalized proposal

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-15 Thread John Roesler
clude the deprecation of: > > > >- KStream#transform > >- KStream#transformValues > >- KStream#flatTransform > >- KStream#flatTransformValues > > > > > > > > On Fri, 11 Feb 2022 at 15:16, John Roesler wrote: > > > >

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-15 Thread John Roesler
My apologies, this feedback was intended for KIP-634. -John On Tue, Feb 15, 2022, at 13:15, John Roesler wrote: > Thanks for the update, Jorge, > > I've just looked over the KIP again. Just one more small > concern: > > 5) We can't just change the type of Record#headers(

Re: [VOTE] KIP-820: Extend KStream process with new Processor API

2022-02-15 Thread John Roesler
Thanks, Jorge! I'm +1 (binding) -John On Tue, 2022-02-15 at 19:16 +, Jorge Esteban Quilcate Otoya wrote: > Hi all, > > I'd like to start a vote on KIP-820 which proposes extending KStream to use > the new Processor API >

Re: [DISCUSS] KIP-634: Complementary support for headers in Kafka Streams DSL

2022-02-15 Thread John Roesler
citly. However, it > > would be good to state whether and when we'll automatically > > wrap a value serde. For example, if the value serde is known > > (or if we're using a default serde from the config), will > > Streams automatically wrap it downstream of the record- > > map

Re: [VOTE] KIP-815: Support max-timestamp in GetOffsetShell

2022-02-23 Thread John Roesler
Thanks for the KIP, Ziming! I'm +1 (binding) -John On Wed, 2022-02-23 at 13:54 +0100, Mickael Maison wrote: > Hi, > > I'm +1 (binding) too. > > Just one minor comment: > Could we make the "time" argument also accept "earliest", "latest" and > "max-timestamp" alongside -1, -2, -3. I think it's

Re: [DISCUSS] KIP-816: Topology changes without local state reset

2022-02-14 Thread John Roesler
e old > > > topology's sub-topology 0' state store A-0004, then what we can do to let > > > the new topology state store to be loaded as the old state store. With > > that > > > in my mind originally, I said maybe option B) is sufficient to rename the > > > dir path /

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-18 Thread John Roesler
specific >> type >> >> would break binary compatibility. >> >> I will update the KIP to reflect this: >> >> >> >>> Modifications to method KStream#process are source compatible with >> >> previous version, though not binary compat

Re: [VOTE} KIP-796: Interactive Query v2

2022-03-22 Thread John Roesler
released for the first time, but after some second thoughts I'm now >> feeling it's okay to take it in a more evolving manner. So I'm preferring >> to keep it and be open for breaking changes in the future. >> >> >> Guozhang >> >> On Tue, Mar 22, 2022 at

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-23 Thread John Roesler
> example above are operations on data. `groupBy()` groups, > > > > > `windowedBy()` > > > > > > > > partitions, `aggregate()` computes the aggregate, `mapValues()` > > > > > > > > maps

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
tting of results be > > > > > controlled by the window level api? If I want to emit results for each > > > > > input record the emit strategy is quite independent from the window. > > > > > So > > > > > I somehow share Matthias' and Guozhang's conc

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
Thanks, Hao! I'm +1 (binding) -John On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote: > Hi all, > > I'd like to start a vote on Kafka Streams KIP-825: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
Thanks, Hao, I’m +1 (binding) -John On Thu, Mar 24, 2022, at 11:38, Hao Li wrote: > Hi all, > > I'd like to start a vote on Kafka Streams KIP-825: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced > > Thanks, > Hao

<    3   4   5   6   7   8   9   10   11   12   >