While working on the follow up PR to deprecate `through()`, I realized that we forgot to add `repartition()` to the Scala API.
I updated the KIP accordingly and my PR also include the update to the Scala API (for adding `repartition()` and deprecating `through()`): https://github.com/apache/kafka/pull/8679 Please let us know if there are any concerns. -Matthias On 5/15/20 10:27 AM, Matthias J. Sax wrote: > Thanks Levani! > > I took the liberty to update the KIP accordingly, because so far no > concerns were raised about the change. > > Also assigned the ticket to myself (let me know if you want to work on > it instead). We should get the change into 2.6 release. > > > -Matthias > > > On 5/15/20 1:34 AM, Levani Kokhreidze wrote: >> Hi Matthias, >> >> Thanks for your thoughts and sorry for taking so long to respond. >> For me that makes total sense, so +1 from my side. >> I took the liberty of creating a ticket for it: >> https://issues.apache.org/jira/browse/KAFKA-10003 >> <https://issues.apache.org/jira/browse/KAFKA-10003> >> >> Regards, >> Levani >> >> >>> On May 12, 2020, at 8:30 PM, Guozhang Wang <wangg...@gmail.com> wrote: >>> >>> Sounds fair to me; I think as a syntax sugar it is a good to have, but >>> sometimes it was "enforced" to be used for repartitioning purposes. >>> >>> On Mon, May 11, 2020 at 7:08 PM Matthias J. Sax <mj...@apache.org> wrote: >>> >>>> As an afterthought to KIP-221, I am wondering if we should deprecate >>>> `KStream#through()`? >>>> >>>> The reasoning is that I assume that most people don't want to manage >>>> topics manually anyway and thus it might be good to guide users to use >>>> repartition(). Furthermore, through() is really just syntactic sugar for >>>> to() followed by builder.stream() (thus people don't really loose >>>> functionality). So far, through() was very nice to have, especially with >>>> PAPI integration in the DSL (users might need to do a manual >>>> repartitioning before transform()) however this pattern should be >>>> subsumed by repartition(). >>>> >>>> Reducing the surface area of our API (instead of just enlarging it) >>>> might be good. >>>> >>>> Thoughts? >>>> >>>> >>>> -Matthias >>>> >>>> On 4/5/20 9:36 PM, John Roesler wrote: >>>>> Thanks for the update, Levani! >>>>> -John >>>>> >>>>> On Sat, Apr 4, 2020, at 04:36, Levani Kokhreidze wrote: >>>>>> Hello, >>>>>> >>>>>> Small update regarding this KIP. As per John’s suggestion during the >>>>>> code review >>>>>> (https://github.com/apache/kafka/pull/7170#discussion_r392608571 >>>>>> <https://github.com/apache/kafka/pull/7170#discussion_r392608571>) >>>>>> we’ve decided to remove KeyValueMapper overloads for the new >>>>>> `repartition` operation for the first release of this feature. >>>>>> Wiki page has been updated accordingly >>>>>> ( >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint >>>> < >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint >>>>> ) >>>>>> >>>>>> Regards, >>>>>> Levani >>>>>> >>>>>>> On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <levani.co...@gmail.com> >>>> wrote: >>>>>>> >>>>>>> Thank you all! >>>>>>> >>>>>>> The vote has been open for ~8 days. KIP has three binding votes (Bill, >>>> Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes! >>>>>>> I’ll mark KIP as accepted and start working on it as soon as possible! >>>>>>> >>>>>>> Regards, >>>>>>> Levani >>>>>>> >>>>>>>> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <matth...@confluent.io> >>>> wrote: >>>>>>>> >>>>>>>> +1 (binding) >>>>>>>> >>>>>>>> On 7/31/19 8:36 AM, Guozhang Wang wrote: >>>>>>>>> Thanks for the update! +1 (binding). >>>>>>>>> >>>>>>>>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze < >>>> levani.co...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hello Guozhang, >>>>>>>>>> >>>>>>>>>> Thanks for the feedback. That’s an interesting point. To be honest, >>>> I >>>>>>>>>> totally missed it. I wasn’t aware that there’s `groupBy` >>>> possibility on >>>>>>>>>> KTable. >>>>>>>>>> I don’t see any reasons why not to add same functionality to KTable >>>>>>>>>> interface. >>>>>>>>>> >>>>>>>>>> I’ve updated the KIP: >>>>>>>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint >>>>>>>>>> < >>>>>>>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint >>>>>>>>>>> >>>>>>>>>> Please let me know if you have any other questions and/or concerns. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Levani >>>>>>>>>> >>>>>>>>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wangg...@gmail.com> >>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hello Levani, >>>>>>>>>>> >>>>>>>>>>> Thanks for the KIP! Just got a quick question here about the >>>> scope: why >>>>>>>>>> do >>>>>>>>>>> we only want this for `KStream`, not `KTable#groupBy` for example? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Guozhang >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bbej...@gmail.com> >>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks for the KIP Levani. >>>>>>>>>>>> >>>>>>>>>>>> +1 (binding) >>>>>>>>>>>> >>>>>>>>>>>> -Bill >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze < >>>>>>>>>> levani.co...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> Still waiting for feedback on this KIP. >>>>>>>>>>>>> Please let me know if you have any concerns and/or questions. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Levani >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman < >>>> sop...@confluent.io >>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Looks good! Thanks Levani, >>>>>>>>>>>>>> >>>>>>>>>>>>>> +1 (non-binding) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sophie >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze < >>>>>>>>>>>>> levani.co...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I’d like to initialize voting on KIP-221: >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint >>>>>>>>>>>>>>> < >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If there’re any more concerns about the KIP, happy to discuss >>>>>>>>>> further. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Levani >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> -- Guozhang >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>> >>>> >>> >>> -- >>> -- Guozhang >> >> >
signature.asc
Description: OpenPGP digital signature
