Re: [DISCUSS] KIP-832 Allow creating a producer/consumer using a producer/consumer config

2022-05-07 Thread John Roesler
changes to apply: >> >>- Create 2 new constructors in KafkaProducer >>- Create a new constructor in KafkaConsumer and increase de visibility >>of an existing one >>- Create a new constructor in TopologyTestDriver >> >> Kr, >> >>

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-09 Thread John Roesler
ohn, > > Long emails are great; responding inline! > > On Sat, May 7, 2022 at 4:54 PM John Roesler wrote: > >> Thanks for the KIP, Jim! >> >> This conversation seems to highlight that the KIP needs to specify >> some of its behavior as well as its APIs, where the be

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-09 Thread John Roesler
f later it is desirable to manage punctuation or standby tasks, then it > should be easy for future folks to modify things. > > Overall, I'd frame this KIP as "pause processing resulting in outputs". > > Cheers, > > Jim > > > >> On Mon, May 9, 2022 a

Re: [VOTE] KIP-832 Allow creating a producer/consumer using a producer/consumer

2022-05-10 Thread John Roesler
+1 (binding) Thanks, François! -John On Tue, May 10, 2022, at 03:09, Bruno Cadonna wrote: > Hi Francois, > > Thanks for the KIP whose link is: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211882578 > > +1 (binding) > > Best, > Bruno > > On 09.05.22 23:23, François Rosière wr

Re: [VOTE] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-10 Thread John Roesler
Thanks Jim, I’m +1 (binding) -John On Tue, May 10, 2022, at 14:05, Jim Hughes wrote: > Hi all, > > I'm asking for a vote on KIP-834: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211882832 > > Thanks in advance! > > Jim

Re: [DISCUSS] Apache Kafka 3.3.0 Release

2022-05-10 Thread John Roesler
+1 from me! Thanks for volunteering, José. -John On Tue, May 10, 2022, at 17:53, José Armando García Sancio wrote: > Hi all, > > I would like to volunteer for the release of Apache Kafka 3.3.0. If > people agree, I'll start working on the release plan and update this > thread. > > Thanks, > -José

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-11 Thread John Roesler
can >> imagine >> > >>>> someone using >> > >>>> the pause feature in such a way is because there is something going >> > >>> wrong, >> > >>>> or about >> > >>>> to go wrong, on t

Re: [HEADS-UP] Modification to KIP Template

2022-05-11 Thread John Roesler
+1 from me also. This is a great idea. Thanks, John On Wed, May 11, 2022, at 02:49, Luke Chen wrote: > Hi Mickael, > > +1 to add "test plan" section to KIP template. > Thanks for the improvement! > > Luke > > On Tue, May 10, 2022 at 11:21 PM Mickael Maison > wrote: > >> Hi, >> >> I did not see

Re: [VOTE] KIP-833: Mark KRaft as Production Ready

2022-05-24 Thread John Roesler
+1 (binding) from me. Thanks, Colin! -John On Tue, May 24, 2022, at 01:58, David Jacot wrote: > +1. Thanks Colin! > > On Tue, May 24, 2022 at 4:50 AM Luke Chen wrote: >> >> +1 from me. >> Kraft is coming!!! >> >> Luke >> >> On Tue, May 24, 2022 at 7:26 AM Israel Ekpo wrote: >> >> > +1 (non-bi

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread John Roesler
Hi Sagar, Thanks for the KIP! I’m not at my computer right now, but I think I confronted a similar problem a while back for the Streams metrics. I think that I already made the “addMetric” method to be idempotent, so if it’s already registered, the call just returns the old one instead of crea

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread John Roesler
;m in favor of your KIP. Also, sorry for responding to the VOTE thread instead of DISCUSS. I'm +1 (binding) -John On Tue, May 24, 2022, at 09:10, John Roesler wrote: > Hi Sagar, > > Thanks for the KIP! > > I’m not at my computer right now, but I think I confronted a similar &

Re: [DISCUSS] KIP-844: Transactional State Stores

2022-05-24 Thread John Roesler
Thanks for the KIP, Alex! I'm really happy to see your proposal. This improvement fills a long-standing gap. I have a few questions: 1. Configuration The KIP only mentions RocksDB, but of course, Streams also ships with an InMemory store, and users also plug in their own custom state stores. I

Re: [DISCUSS] KIP-846: Task-level Streams metrics for bytes/records Produced

2022-05-28 Thread John Roesler
Thanks for the well motivated and documented KIP, Sophie! I’m in favor of this change. -John On Sat, May 28, 2022, at 06:42, Sophie Blee-Goldman wrote: > Hey all, > > I'd like to propose a very small KIP to add two metrics that will help fill > a gap in the derivable produced and consumed metri

Re: [VOTE] KIP-846: Processor-level Streams metrics for records/bytes Producedd

2022-05-31 Thread John Roesler
+1 (binding) Thanks, John On Mon, May 30, 2022, at 13:00, Bill Bejeck wrote: > +1 (binding) > > -Bill > > On Mon, May 30, 2022 at 4:49 AM Sagar wrote: > >> +1 (non-binding). >> >> Thanks! >> Sagar. >> >> On Mon, May 30, 2022 at 1:11 PM Bruno Cadonna wrote: >> >> > +1 (binding) >> > >> > Thanks!

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-31 Thread John Roesler
Generally, I agree with Ismael that having a new, weird name will make it hard to keep them straight. Then again, we need to make them different to prevent confusion about their semantics. To be clear, I'll be a +1 regardless of how we break this dilemma. One suggestion: We currently have addMe

Re: [VOTE] KIP-827: Expose logdirs total and usable space via Kafka API

2022-05-31 Thread John Roesler
Thanks for the KIP Mickael, I'm +1 (binding) -John On Tue, May 31, 2022, at 18:48, Jun Rao wrote: > Hi, Mickael, > > Thanks for the KIP. +1 > > Jun > > On Wed, May 25, 2022 at 7:54 AM Tom Bentley wrote: > >> Hi Mickael, >> >> Thanks for the KIP! +1 (binding). >> >> Kind regards, >> >> Tom >> >>

Re: [VOTE] KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer

2022-06-08 Thread John Roesler
Thanks for the KIP, Alexandre! I’m +1 (binding) -John On Wed, Jun 8, 2022, at 20:54, deng ziming wrote: > Thank you for this KIP, > > +1 (non-binding) > > -- > Best, > Ziming > >> On Jun 7, 2022, at 8:53 PM, Alexandre Garnier wrote: >> >> Hi! >> >> A little reminder to vote for this KIP. >>

Re: [VOTE] KIP-851: : Add requireStable flag into ListConsumerGroupOffsetsOptions

2022-06-30 Thread John Roesler
Thanks for the KIP, Guozhang! I’m +1 (binding) -John On Thu, Jun 30, 2022, at 21:17, deng ziming wrote: > Thanks for this KIP, > we have a kafka-consumer-groups.sh shell which is based on the API you > proposed to change, is it worth update it as well? > > -- > Best, > Ziming > >> On Jul 1, 202

Re: [DISCUSS] Website changes required for Apache projects

2022-07-20 Thread John Roesler
Hi all, Yes, thanks Divij for driving this! I tend to agree with Mickael about having vendor branding front-and-center like that. On the other hand, I think the video itself is quite nice, and it's a good thing to put in front of newcomers for a human introduction to the project. I took a look

Re: [DISCUSS] Website changes required for Apache projects

2022-07-21 Thread John Roesler
should be the same as > accepting code changes i.e. it goes through a community review requiring > votes committers/PMC members. > > Regards, > Divij Vaidya > > > > On Thu, Jul 21, 2022 at 3:57 AM John Roesler wrote: > >> Hi all, >> >> Yes, thank

Re: [DISCUSS] Website changes required for Apache projects

2022-07-28 Thread John Roesler
>> > > >> What do you think? >> > > >> >> > > >> >> > > >> On Thu, Jul 21, 2022 at 4:19 AM Divij Vaidya < >> divijvaidy...@gmail.com >> > > >> > > >> wrote: >> > > >> &

Re: [ANNOUNCE] New committer: Lucas Bradstreet

2023-02-16 Thread John Roesler
Congratulations, Lucas! -John On Thu, Feb 16, 2023, at 19:52, Luke Chen wrote: > Congratulations, Lucas! > > Luke > > On Fri, Feb 17, 2023 at 8:57 AM Guozhang Wang > wrote: > >> Congratulations, Lucas! >> >> On Thu, Feb 16, 2023 at 3:18 PM Kowshik Prakasam >> wrote: >> > >> > Congratulations, Lu

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-23 Thread John Roesler
Thanks for the KIP, Philip! I think your proposal makes sense. I suspect the reason that we previously did the DNS resolution in the constructor is to fail fast if the name is wrong. On the other hand, it's generally a hassle to do failure-prone or slow operations in a constructor, so I'm in fa

Re: [VOTE] KIP-898: Modernize Connect plugin discovery

2023-02-28 Thread John Roesler
Thanks for the KIP, Greg! I’m +1 (binding) I really appreciate all the care you took in the migration and test design. Thanks, John On Tue, Feb 28, 2023, at 04:33, Federico Valeri wrote: > +1 (non binding) > > Thanks > Fede > > On Tue, Feb 28, 2023 at 10:10 AM Mickael Maison > wrote: >> >> +1

Re: [ANNOUNCE] New Kafka PMC Member: David Arthur

2023-03-09 Thread John Roesler
Congratulations, David! -John On Thu, Mar 9, 2023, at 20:18, ziming deng wrote: > Congrats David! > > Ziming > >> On Mar 10, 2023, at 10:02, Luke Chen wrote: >> >> Congratulations, David! >> >> On Fri, Mar 10, 2023 at 9:56 AM Yash Mayya wrote: >> >>> Congrats David! >>> >>> On Thu, Mar 9, 20

Re: [ANNOUNCE] New Kafka PMC Member: Chris Egerton

2023-03-09 Thread John Roesler
Congratulations, Chris! -John On Thu, Mar 9, 2023, at 20:02, Luke Chen wrote: > Congratulations, Chris! > > On Fri, Mar 10, 2023 at 9:57 AM Yash Mayya wrote: > >> Congratulations Chris! >> >> On Thu, Mar 9, 2023, 23:42 Jun Rao wrote: >> >> > Hi, Everyone, >> > >> > Chris Egerton has been a Kafka

Re: [DISCUSS] KIP-914 Join Processor Semantics for Versioned Stores

2023-03-09 Thread John Roesler
Thanks for the KIP, Victoria! I had some questions/concerns, but you addressed them in the Rejected Alternatives section. Thanks for the thorough proposal! -John On Thu, Mar 9, 2023, at 18:59, Victoria Xia wrote: > Hi everyone, > > I have a proposal for updating Kafka Streams's stream-table joi

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-03-24 Thread John Roesler
Thanks for the KIP, Sagar! At first glance, this seems like a very useful feature. A common pain point in Streams is when upstream producers don't send regular updates and stream time cannot advance. This causes stream-time-driven operations to appear to hang, like time windows not closing, sup

Re: [VOTE] KIP-906 Tools migration guidelines

2023-03-24 Thread John Roesler
Thanks Federico, I'm +1 (binding) -John On Fri, Mar 24, 2023, at 01:11, Federico Valeri wrote: > Bumping this thread for more votes. > > Thanks. > > On Wed, Mar 15, 2023, 9:57 AM Alexandre Dupriez > wrote: > >> Hi, Frederico, >> >> Thanks for the KIP. >> >> Non-binding +1. >> >> Thanks, >> Alex

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-04-28 Thread John Roesler
Hi all, This is a great suggestion! It seems like a really good way to make the Apache Kafka project more efficient in general and also smooth the path to committership. I've brought the topic up with the Apache Kafka PMC to consider adopting a policy around the "collaborator" rule. In the me

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-05-12 Thread John Roesler
Thanks again for bringing this up, David! As an update to the community, the PMC has approved a process to make use of this feature. Here are the relevant updates: PR to add the policy: https://github.com/apache/kafka-site/pull/510 PR to update the list: https://github.com/apache/kafka/pull/13

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-05-15 Thread John Roesler
, Divij Vaidya, Lucas Brutschy, Yash Mayya, Philip Nee, vamossagar12, Christo Lolov, Federico Valeri, and andymg3 Thanks all, -John On 2023/05/12 15:53:40 John Roesler wrote: > Thanks again for bringing this up, David! > > As an update to the community, the PMC has approved a process to

Re: [VOTE] KIP-927: Improve the kafka-metadata-quorum output

2023-05-20 Thread John Roesler
I’m +1 (binding) Thanks, Federico! It looks like a nice improvement to me. -John On Sat, May 20, 2023, at 09:16, Kamal Chandraprakash wrote: > +1 (non binding) > > On Wed, May 17, 2023 at 3:22 AM Divij Vaidya > wrote: > >> +1 (non binding) >> >> Divij Vaidya >> >> >> >> On Tue, May 16, 2023

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

2023-05-22 Thread John Roesler
Hi Hao, Thanks for the KIP! Overall, I think this is a great idea. I always wanted to circle back after the Smooth Scaling KIP to put a proper optimization algorithm into place. I think this has the promise to really improve the quality of the balanced assignments we produce. Thanks for provi

Re: [VOTE] KIP-925: rack aware task assignment in Kafka Streams

2023-06-05 Thread John Roesler
Thanks, Hao! I'm +1 (binding) -John On 2023/06/05 18:07:56 Walker Carlson wrote: > +1 (binding) > > On Mon, Jun 5, 2023 at 3:14 AM Bruno Cadonna wrote: > > > Hi Hao, > > > > +1 (binding) > > > > Thanks! > > Bruno > > > > On 30.05.23 21:16, Colt McNealy wrote: > > > +1 (non-binding) > > > > > >

Re: [VOTE] KIP-923: Add A Grace Period to Stream Table Join

2023-06-06 Thread John Roesler
Thanks for the KIP, Walker! I’m +1 (binding) Thanks, John On Mon, Jun 5, 2023, at 13:39, Victoria Xia wrote: > Hi Walker, > > Thanks for the KIP! Left a clarification question on the discussion thread > just now but it's about an implementation detail, so I don't think it > changes anything in t

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-06 Thread John Roesler
, May 24, 2023 at 4:54 AM Federico Valeri < >> > > > fedeval...@gmail.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > +1 on Divij suggestions >> > > > > > > > >> > &g

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-06 Thread John Roesler
ing with the assumption that things are working for me even if I > didn't have to accept an invite. Is that a correct assumption? > > -- > Divij Vaidya > > > > On Tue, Jun 6, 2023 at 5:32 PM John Roesler wrote: > >> Hello all, >> >> I put in a ticket

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-06 Thread John Roesler
Hello Greg, Andrew, and Victoria, I have just re-added you to the file, so you should receive new invites. Please let us know if you have any trouble! Thanks, -John On Tue, Jun 6, 2023, at 10:31, John Roesler wrote: > Hello all, > > I put in a ticket with Apache Infra to re-send thes

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-07 Thread John Roesler
n 6, 2023 at 8:32 AM John Roesler wrote: > >> Hello all, >> >> I put in a ticket with Apache Infra to re-send these invites, and they >> told me I should just remove the usernames in one commit and then re-add >> them in a subsequent commit to trigger the invites again

Re: [VOTE] 3.5.0 RC1

2023-06-07 Thread John Roesler
Thanks for running this release, Mickael! I've verified: * the signature * that I can compile the project * that I can run the tests. I saw one flaky test failure, but I don't think it should block us. Reported as https://issues.apache.org/jira/browse/KAFKA-13531?focusedCommentId=17730190 * the

Re: [ANNOUNCE] New committer: Divij Vaidya

2023-06-13 Thread John Roesler
Congratulations, Divij! -John On Tue, Jun 13, 2023, at 09:44, David Arthur wrote: > Congrats Divij! > > On Tue, Jun 13, 2023 at 12:34 PM Igor Soarez wrote: > >> Congratulations Divij! >> >> -- >> Igor >> > > > -- > -David

Re: [VOTE] KIP-872: Add Serializer#serializeToByteBuffer() to reduce memory copying

2023-06-20 Thread John Roesler
Hi Divij and ShunKang, I pulled open this thread to see if you needed my vote, but FYI, Divij is a committer now, so he can re-cast his vote as binding. Thanks, -John On 2023/06/20 13:37:04 ShunKang Lin wrote: > Hi all, > > Bump this thread again and see if we could get a few more votes. > Cur

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
nced that it's worth having both transactional/non-transactional stores available, as it would considerably increase the complexity of the codebase, for very little benefit. 4. Method deprecation: Are you referring to StateStore#getPosition()? As I understand it, Position cont

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
we're already waiting for acks and a RocksDB commit. Thanks, -John On 6/20/23 14:09, Nick Telford wrote: Hi John, By "RecordCache", do you mean the RocksDB "WriteBatch"? I can't find any class called "RecordCache"... Cheers, Nick On Tue, 20 Jun 2023

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
ngest them on commit, when we're already waiting for acks and a RocksDB commit. In this case, how would uncommitted records be read by joins? On Tue, 20 Jun 2023, 20:51 John Roesler, wrote: Ah, sorry Nick, I just meant the regular heap based cache that we maintain in Streams. I see tha

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
uld just go the memorable with Rocks WriteBatches route if the cache is disabled? On Tue, 20 Jun 2023, 22:00 John Roesler, wrote: Touché! Ok, I agree that figuring out the case of a disabled cache would be non-trivial. Ingesting single-record SST files will probably not be performant, but benchma

Re: [DISCUSS] KIP-941: Range queries to accept null lower and upper bounds

2023-06-21 Thread John Roesler
Hi all, Thanks for the KIP, Lucia! This is a nice change. To Kirk's question (1), the example is a bit misleading. The typical case that would ease user pain is specifically using "null" to indicate an open-ended range, especially since null is not a valid key. I could additionally see an em

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-21 Thread John Roesler
Thanks Nick, That sounds good to me. I can't let (2) slide, though.. Writing and ingesting SST files is not a RocksDB internal, but rather a supported usage pattern on public APIs. Regardless, I think your overall preference is fine with me, especially if we can internalize this change within

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-21 Thread John Roesler
ng. On Wed, 21 Jun 2023 at 18:25, John Roesler wrote: Thanks Nick, That sounds good to me. I can't let (2) slide, though.. Writing and ingesting SST files is not a RocksDB internal, but rather a supported usage pattern on public APIs. Regardless, I think your overall preference is fine wi

Re: [VOTE] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2021-01-19 Thread John Roesler
or. > > > > > > > > > > > > > > I actually would have advocated to remove this config entirely, > > but > > > as > > > > > John > > > > > > > pointed > > > > > > > out, we still n

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-01-22 Thread John Roesler
andle+window+size > > Thanks, > Leah > > On Thu, Jan 14, 2021 at 9:36 AM John Roesler wrote: > > > Hi David, > > > > Thanks for the heads-up; it's added. > > > > -John > > > > On Thu, 2021-01-14 at 08:43 +0100, David Jacot wrot

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

2021-01-27 Thread John Roesler
Hi Levani, Thanks for this KIP! I think this is really high value; it was something I was disappointed I didn’t get to do as part of KIP-441. Rack awareness is a feature provided by other distributed systems as well. I wonder if your KIP could devote a section to summarizing what rack awareness

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-01-27 Thread John Roesler
marev wrote: > Hi John, > > KIP-418 is already implemented and reviewed, but I don't see it in the > release plan. Can it be added? > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-418%3A+A+method-chaining+way+to+branch+KStream > > Regards, > > Ivan &

Re: [VOTE] KIP-698: Add Explicit User Initialization of Broker-side State to Kafka Streams

2021-01-28 Thread John Roesler
hould be > > > > > > similar, I am fine with option b). However, IMO option a) is a bit > > > > > > cleaner. WDYT? > > > > > > > > > > > > > 2) Since `init()` is a blocking call (we only return after all > > >

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

2021-01-28 Thread John Roesler
he great work on KIP441 :) ). > Makes sense, will add a section in the KIP explaining rack awarenesses on > high level and how it’s implemented in the different distributed systems. > > Thanks, > Levani > > > On 27. Jan 2021, at 16:07, John Roesler wrote: > > >

Re: [DISCUSS] Apache Kafka 2.6.2 release

2021-01-28 Thread John Roesler
Thanks so much for stepping up, Sophie! I'm +1 -John On Wed, 2021-01-27 at 17:59 -0500, Bill Bejeck wrote: > Thanks for taking this on Sophie. +1 > > Bill > > On Wed, Jan 27, 2021 at 5:59 PM Ismael Juma wrote: > > > Thanks Sophie! +1 > > > > Ismael > > > > On Wed, Jan 27, 2021 at 2:45 PM S

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

2021-02-01 Thread John Roesler
or > and let users add their own implementation for them separately if they > like via config. > > If this approach sounds reasonable, I’ll work on updating KIP this week. > > Thanks, > Levani > > > On 28. Jan 2021, at 19:20, John Roesler wrote: > > &

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

2021-02-02 Thread John Roesler
pluggable assignor > can be differed to another KIP, yes. > As it won’t be required for this KIP and use-cases I had in mind to > work. > > Regards, > Levani > > > > On 2. Feb 2021, at 07:55, John Roesler wrote: > > > > Hello Levani, > > > &

Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-03 Thread John Roesler
e is whether we should support this apparently artificial (testing-only) use case to the point where it's worth adding a whole new method to the Consumer interface. Thanks all, John On Thu, 2020-12-17 at 13:18 -0600, John Roesler wrote: > Thanks Jason, > > We would only return t

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-03 Thread John Roesler
reeze on Feb 17th. Thanks as always for all of your contributions, John On Wed, 2021-01-27 at 12:17 -0600, John Roesler wrote: > Hello again, all. > > This is a reminder that *today* is the KIP freeze for Apache > Kafka 2.8.0. > > The next checkpoint is the Feature Freez

Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-04 Thread John Roesler
sOrMetadata(Duration) or something. > > > > > > > > It seems unreliable to expect the broker to return a > > > > particular record within a particular timeout in general, > > > > which is what these tests are doing. The broker can decide > >

Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-05 Thread John Roesler
turn_on_response > > > > This idea LGTM. It not only makes minimum changes to current behavior but > > also works for KIP-695. > > > > On 2021/02/04 16:07:11, John Roesler wrote: > > > Hi Matthias, Chia-Ping, and Tom, > > > > > >

New release branch 2.8

2021-02-07 Thread John Roesler
Hello Kafka developers and friends, As promised, we now have a release branch for the 2.8.0 release. The branch is "2.8" (https://github.com/apache/kafka/tree/2.8) . Trunk has been bumped to 2.9.0-SNAPSHOT. I'll be going over the JIRAs to move every non-blocker from this release to the next rele

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-07 Thread John Roesler
, including resolving failures, writing new tests, and fixing documentation. Thanks as always for your contributions, John On Wed, 2021-02-03 at 14:18 -0600, John Roesler wrote: > Hello again, all, > > This is a reminder that today is the Feature Freeze > deadline. To avoid any last-minute cr

Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-10 Thread John Roesler
lso like the `PollOptions` idea, but I have to agree that the config > > option would be the least disruptive approach. > > > > Thanks, > > Bill > > > > On Fri, Feb 5, 2021 at 6:12 PM John Roesler wrote: > > > > > Thanks, all! > > > >

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-16 Thread John Roesler
the specific > > option we're discussing here, but it may be more useful for other options > > we may add in the future > > > > Just my 2 cents. But if the pollOptions proposal would really add so much > > additional work that it would cause the 2.8 release to be sign

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-18 Thread John Roesler
you have forgotten to update >the docs for some features in 2.8.0. Once we have a green build and passing system tests, I will cut the first RC. Thank you, John On Sun, Feb 7, 2021, at 09:59, John Roesler wrote: > Hello all, > > I have just cut the branch for 2.8 and sent the notifi

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-21 Thread John Roesler
rokers > > The PRs for these have already been approved and merged to the `trunk` > branch (and backported to a few older branches). I'd like to backport these > to the `2.8` branch. > > Thanks, and best regards! > > Randall > > On Thu, Feb 18, 2021 at 10:23 AM J

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
; open a room to carry more information without adding more Public APIs to > Consumer. > > On 2021/02/17 05:21:06, John Roesler wrote: > > Hello again, all, > > > > Thank you for your feedback and patience. I am hopeful that > > I have been able to come up with a sol

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-02-22 Thread John Roesler
tion in the User Manual at > > This error obstructs us from running integration tests so I'd like to push it > to 2.8 branch after it gets approved. > > Best Regards, > Chia-Ping > > On 2021/02/18 16:23:13, "John Roesler" wrote: > > Hello again, all.

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
Hello all, Since there haven't been any big objections to my proposed design fix, I'm updating the KIP now and opening the PR for reviews. Hopefully, we can get this merged quickly and unblock the system tests. Thanks, John On Mon, 2021-02-22 at 09:49 -0600, John Roesler wrote: &g

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
to provide information the > consumer has access to when it receives the fetch response. > > Thanks again. > > Bill > > On Mon, Feb 22, 2021 at 10:51 AM John Roesler wrote: > > > Hello all, > > > > Since there haven't been any big objections to my

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

2021-02-22 Thread John Roesler
much to you all for your help on this! John On Mon, 2021-02-22 at 10:25 -0600, John Roesler wrote: > Thanks, Bill, > > I was waiting for feedback on the "currentLag" proposal > before updating the KIP. Since there haven't been any > objections to my new proposal, I&

Re: [DISCUSS] Apache Kafka 2.8.0 release

2021-03-02 Thread John Roesler
nk. > > - Dhruvil > > On Mon, Feb 22, 2021 at 7:50 AM John Roesler wrote: > > > Thanks for the heads-up, Chia-Ping, > > > > I agree it would be good to include that fix. > > > > Thanks, > > John > > > > On Mon, 2021-02-22 at 09

Re: [VOTE] KIP-708: Rack awareness for Kafka Streams

2021-03-11 Thread John Roesler
Thanks for the KIP! I'm +1 (binding) -John On Wed, 2021-03-10 at 13:13 +0200, Levani Kokhreidze wrote: > Hello all, > > I’d like to start the voting on KIP-708 [1] > > Best, > Levani > > [1] - > https://cwiki.apache.org/confluence/display/KAFKA/KIP-708%3A+Rack+awareness+for+Kafka+Streams >

Re: [DISCUSS] Apache Kafka 2.7.1 release

2021-03-25 Thread John Roesler
Hi Mickael, I hope all is well with you. FYI, I have just pushed the fix to this blocker: https://issues.apache.org/jira/browse/KAFKA-12508 to the 2.7 branch. I'll resolve the ticket after I finish pushing to 2.6. Thanks, John On Fri, 2021-02-19 at 05:29 -0800, Ismael Juma wrote: > Sounds good,

[VOTE] 2.8.0 RC0

2021-03-30 Thread John Roesler
Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 2.8.0. This is a major release that includes many new features, including: * Early-access release of replacing Zookeeper with a self- managed quorum * Add Describe Cluster API * Support mu

Re: [VOTE] 2.8.0 RC0

2021-03-30 Thread John Roesler
2.8.0 release, but wanted to get RC0 out asap for testing. Thank you, John On Tue, 2021-03-30 at 16:37 -0500, John Roesler wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of Apache Kafka > 2.8.0. This is a major release that

Re: [VOTE] 2.6.2 RC0

2021-03-31 Thread John Roesler
Hi Justin, Thanks for bringing this up and for your advice in resolving it. I've carved out the specific chunk of work relating to copyright, and wanted to call your attention to a specific proposal I have to fix this situation: https://issues.apache.org/jira/browse/KAFKA-12593?focusedCommentId=17

Re: [DISCUSS] KIP-633: Drop 24 hour default of grace period in Streams

2021-03-31 Thread John Roesler
Thanks for bringing this up, Sophie! This has indeed been a pain point for a lot of people. It's a really thorny issue with no obvious "right" solution. I think your proposal is a good one. Thanks, -John On Wed, 2021-03-31 at 13:28 -0700, Sophie Blee-Goldman wrote: > Hey all, > > It's finally

[DISCUSS] Please review 2.8.0 blog post

2021-04-01 Thread John Roesler
Hello all, In the steady march toward the Apache Kafka 2.8.0 release, I have prepared a draft of the release announcement post: https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache5 If you have a moment, I would greatly appreciate your reviews. Thank you, -John

Re: [DISCUSS] KIP-725: Streamlining configurations for TimeWindowedDeserializer.

2021-04-01 Thread John Roesler
Hi Sagar, I think this is a good proposal. The only change I might recommend is to change the config to more closely align with the other one, for example: “window.inner.class.deserializer” But it’s very minor; I leave it to your judgement. Thanks, John On Fri, Mar 26, 2021, at 03:36, Sagar

Re: [DISCUSS] KIP-718: Make KTable Join on Foreign key unopinionated

2021-04-01 Thread John Roesler
ang, > > Apologies for the late answer. Originally that was my proposal - to > piggyback on the provided materialisation method ( > https://issues.apache.org/jira/browse/KAFKA-10383). > John Roesler suggested to us to provide even further fine tuning on API > level parameters. Maybe

Re: [VOTE] KIP-725: Streamlining configurations for TimeWindowedDeserializer.

2021-04-05 Thread John Roesler
Thanks, Sagar! I’m +1 (binding) -John On Mon, Apr 5, 2021, at 21:35, Sophie Blee-Goldman wrote: > Thanks for the KIP! +1 (binding) from me > > Cheers, > Sophie > > On Mon, Apr 5, 2021 at 7:13 PM Sagar wrote: > > > Hi All, > > > > I would like to start voting on the following KIP: > > > > htt

Re: [VOTE] KIP-633: Drop 24 hour default of grace period in Streams

2021-04-05 Thread John Roesler
Thanks, Sophie, I’m +1 (binding) -John On Mon, Apr 5, 2021, at 21:34, Sophie Blee-Goldman wrote: > Hey all, > > I'd like to start the voting on KIP-633, to drop the awkward 24 hour grace > period and improve the API to raise visibility on an important concept in > Kafka Streams: grace period na

Re: [DISCUSS] Please review 2.8.0 blog post

2021-04-05 Thread John Roesler
r. > > On Mon, Apr 5, 2021 at 6:29 PM Adam Bellemare > wrote: > > > Read it all. It looks good to me in terms of structure and content. I am > > not sufficiently up to date on all the features that are otherwise included > > in 2.8.0, but the ones listed seem very promi

Re: [DISCUSS] KIP-718: Make KTable Join on Foreign key unopinionated

2021-04-06 Thread John Roesler
owed-scoped MaterializedSubscription since it seems we want to allow > > users to fully customize this store? > > > > Guozhang > > > > On Thu, Apr 1, 2021 at 5:28 PM John Roesler wrote: > > > > > Thanks Marco, > > > > > > Sorry

Subject: [VOTE] 2.8.0 RC1

2021-04-06 Thread John Roesler
Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 2.8.0. This is a major release that includes many new features, including: * Early-access release of replacing Zookeeper with a self- managed quorum * Add Describe Cluster API * Support m

Re: [VOTE] 2.8.0 RC0

2021-04-06 Thread John Roesler
Hello again, all, I am closing this vote in favor of the 2.8.0 RC1 thread. Thank you, John On Tue, 2021-03-30 at 17:11 -0500, John Roesler wrote: > Hello again, all, > > I just wanted to mention that I am aware of Justin's > concerns in the 2.6.2 thread: > https://lists.ap

Re: [DISCUSS] Please review 2.8.0 blog post

2021-04-07 Thread John Roesler
for the explanation John! > > > > Thanks! > > Sagar. > > > > On Tue, Apr 6, 2021 at 8:16 AM John Roesler wrote: > > > > > Oh, my apologies, Sagar, > > > > > > That link will not resolve until the release. The release >

Re: [kafka-clients] Subject: [VOTE] 2.8.0 RC1

2021-04-07 Thread John Roesler
ad. > > Could we assume that was just a carry over from the previous solicitation > for 2.8.0 RC0 and you actually meant to say 2021-04-13? > > When you have a moment, please clarify. > > I am running my tests shortly and will share my results by the end of the > week. &g

Re: I have a question about apache kafka.

2021-04-08 Thread John Roesler
Hello Yun Han Nam, The users@ list is more typical for this kind of topic, but this list is fine, too. What’s the question? Thanks, John On Thu, Apr 8, 2021, at 06:25, 남윤한[Yun Han Nam] wrote: > Hi. > > I want to ask you a question about Apache Kafka, is this the right one? > > It is a techni

Re: [VOTE] KIP-725: Streamlining configurations for TimeWindowedDeserializer.

2021-04-08 Thread John Roesler
r, +1 non-binding. Thanks again for doing this. > > > > > > Leah > > > > > > On Mon, Apr 5, 2021 at 9:40 PM John Roesler wrote: > > > > > > > Thanks, Sagar! > > > > > > > > I’m +1 (binding) > > >

Re: Subject: [VOTE] 2.8.0 RC1

2021-04-12 Thread John Roesler
te-docs haven't been > installed to https://kafka.apache.org/28/documentation.html yet. > > Thanks again! > > -Bill > > On Tue, Apr 6, 2021 at 5:37 PM John Roesler wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is th

Re: [VOTE] KIP-718: Make KTable Join on Foreign key unopinionated

2021-04-14 Thread John Roesler
I’m +1 (binding) Thanks, Marco! -John On Wed, Apr 14, 2021, at 07:59, Marco Aurélio Lotz wrote: > Hi all, > > I would like to start a vote on KIP-718, which proposes to make KTable join > on foreign key unopinionated in terms of persistence method (currently it > forces RocksDB usage for subscri

Re: [DISCUSS] KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2

2021-04-14 Thread John Roesler
Thanks for the KIP, Sophie. I think this is a great approach, and it makes perfect sense in the 3.0 release as well. -John On Wed, Apr 14, 2021, at 01:17, Bruno Cadonna wrote: > Hi Sophie, > > the KIP makes sense to me! Thank you for writing it! > > Best, > Bruno > > On 14.04.21 04:52, Soph

Re: [DISCUSS] KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2

2021-04-14 Thread John Roesler
Hello again, Sophie, I've just had a late-breaking thought about your KIP. It might be worthwhile also printing a WARN log when the deprecated configs are used. People may not always be referencing the StreamsConfig fields in their source code when they set these configs, so it would be good to ma

[VOTE] 2.8.0 RC2

2021-04-14 Thread John Roesler
Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 2.8.0.This is a major release that includes many new features, including: * Early-access release of replacing Zookeeper with a self-managed quorum * Add Describe Cluster API * Suppor

Re: [kafka-clients] Re: Subject: [VOTE] 2.8.0 RC1

2021-04-14 Thread John Roesler
n, Apr 12, 2021 at 8:47 PM John Roesler > wrote: > > Good catch, Israel! > > > > I’ll make sure that gets fixed. > > > > Thanks, > > John > > > > On Mon, Apr 12, 2021, at 19:30, Israel Ekpo wrote: > > > I just noticed that with the la

<    3   4   5   6   7   8   9   10   11   12   >