Kip Write Access

Hello, I wish to write a kip. Could you grant me access? Thanks (Wiki username is yohan.richard.yu)

KIP-202

Hi, Please take a look at: https://cwiki.apache.org/confluence/display/KAFKA/KIP-202+Move+merge%28%29+from+StreamsBuilder+to+KStream Thanks

Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

od does not use > varargs anymore but a single KStream parameter (in contrast to the old > method). And mention that this is no limitation as calls to new merge() > can be chained. > > > > Thanks a lot! > > -Matthias > > > > On 9/17/17 10:32 AM, Richard Yu wrote: > > Correction:

Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

> > > Hi Richard, > > > > > > Thanks for the KIP. Looks good, just one thing: we don't need to > > deprecate > > > StreamBuilder#merge as it has been added during this release cycle. It > > can > > > just be removed. > > > > > > Th

Re: [VOTE] KIP-202

an exception from this? > > > -Matthias > > On 9/19/17 7:09 AM, Richard Yu wrote: > > Kip has been changed to suit 1.0.0 release. > > > > On Tue, Sep 19, 2017 at 6:24 AM, Damian Guy <damian@gmail.com> > wrote: > > > >> +1 > >> &g

Re: [VOTE] KIP-202

; if we will add > it > > in > > > 1.1.0, then we indeed need to deprecate it. > > > > > > > > > Guozhang > > > > > > > > > On Tue, Sep 19, 2017 at 7:29 AM, Richard Yu < > yohan.richard...@gmail.com> > > > wrote: &g

[Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

Hi, Please take a look at: https://cwiki.apache.org/confluence/display/KAFKA/KIP- 202+Move+merge%28%29+from+StreamsBuilder+to+KStream Thanks

Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

KIP-202 has been changed according to the conditions of your suggestion. On Sun, Sep 17, 2017 at 8:51 AM, Richard Yu <yohan.richard...@gmail.com> wrote: > I added StreamsBuilder under the assumption that InternalStreamBuilder > would be required to merge > two s

Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

of the old merge() method. On Sun, Sep 17, 2017 at 9:37 AM, Richard Yu <yohan.richard...@gmail.com> wrote: > With regards to Xavier's comment, this practice I do no think applies to > this PR. There is not much potential here for warnings to be thrown. Note > that in Streams

Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

un, Sep 17, 2017 at 9:10 AM, Richard Yu <yohan.richard...@gmail.com> wrote: > KIP-202 has been changed according to the conditions of your suggestion. > > On Sun, Sep 17, 2017 at 8:51 AM, Richard Yu <yohan.richard...@gmail.com> > wrote: > >> I added StreamsBuilder unde

Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

.merge(stream2); > > > Having pointed out the second pattern, it should actually be fine to get > rid of varargs in merger() at all, as users could chain multiple calls > to merge() after each other: > > KStream multiMerged = stream1.merge(s2).merge(s3).merge(s4); > > &

Re: [VOTE] KIP-202

KIP-202 Move merge() from StreamsBuilder to KStream. https://cwiki.apache.org/confluence/display/KAFKA/KIP-202+Move+merge%28%29+from+StreamsBuilder+to+KStream This is the link for the VOTE. On Mon, Sep 18, 2017 at 4:27 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Hello, I w

[VOTE] KIP-202

Hello, I would like to start a VOTE thread on KIP-202. Thanks.

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

could we rename `keys` to > > `all` and `all` to `range` to be consistent with the other store's APIs? > > > > 2. One meta comment on the implementation details: since both `keys` and > > `all` would likely touch multiple segments, we may need to use the

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

ees when using those APIs > with regard to consistency -- not saying that we need to provide strong > guarantees, but he KIP should describe what user can expect. > > > -Matthias > > On 9/24/17 8:11 PM, Richard Yu wrote: > > Hello, I would like to solicit review and commen

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Léauté <xav...@confluent.io> wrote: > Thank you Richard! Do you or Guozhang have any thoughts on my suggestions > to use fetchAll() and fetchAll(timeFrom, timeTo) and reserve the "range" > keyword for when we query a specific range of keys? > > Xavier > > On Sa

Re: [VOTE] KIP-202

Ismael Juma <isma...@gmail.com> wrote: > > > Thanks for the KIP, +1 (binding). > > > > On 19 Sep 2017 12:27 am, "Richard Yu" <yohan.richard...@gmail.com> > wrote: > > > > > Hello, I would like to start a VOTE thread on KIP-202. > > > > > > Thanks. > > > > > > > > > -- > -- Guozhang >

[DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Hello, I would like to solicit review and comment on this issue (link below): https://cwiki.apache.org/confluence/display/KAFKA/KIP-205%3A+Add+getAllKeys%28%29+API+to+ReadOnlyWindowStore

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Is this KIP close to completion? Because we could start working on the code itself now. (Its at about this stage). On Mon, Oct 16, 2017 at 7:37 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > As Guozhang Wang mentioned earlier, we want to mirror the structure of > similar

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Soliciting more feedback before vote. On Wed, Oct 18, 2017 at 8:26 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Is this KIP close to completion? Because we could start working on the > code itself now. (Its at about this stage). > > On Mon, Oct 16, 2017 at 7

[VOTE] KIP-205: Add all() and range() API to ReadOnlyWindowStore

Hi all, I want to propose KIP-205 for the addition of new API. It is about adding methods similar to those found in ReadOnlyKeyValueStore to the ReadOnlyWindowStore class. As it appears the discussion has reached a conclusion, I would like to start the voting process.

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Xavier: There has been two pluses on the voting thread. Are you fine with the current formation? On Tue, Oct 24, 2017 at 4:26 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > I think we can come up with this compromise: range(long timeFrom, long > timeTo) will be changed to

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

I think we can come up with this compromise: range(long timeFrom, long timeTo) will be changed to getKeys(long timeFrom, long timeTo). Sounds fair? On Tue, Oct 24, 2017 at 10:44 AM, Xavier Léauté wrote: > > > > Generally I think having `all / range` is better in terms of

[VOTE] KIP-266: Add TimeoutException to KafkaConsumer#position()

Hi all, It appears that discussion is coming to a close for KIP-266. I would like to start a voting thread for this KIP. Here is the link for reference. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75974886 Thanks, Richard

Re: [VOTE] KIP-266: Add TimeoutException for KafkaConsumer#position

Hi all, I would like to bump this thread since discussion in the KIP appears to be reaching its conclusion. On Thu, Mar 15, 2018 at 3:30 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Hi all, > > Since there does not seem to be too much discussion in KIP-266, I wil

Re: [VOTE] KIP-266: Add TimeoutException for KafkaConsumer#position

..@confluent.io> > > >>>> wrote: > > >>>> > > >>>>> Thanks for the KIP, +1 (binding). > > >>>>> > > >>>>> One small correction: the KIP mentions that close() will be > > >> deprecated, &

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

; > > > > > > > > On Thu, Apr 19, 2018 at 12:12 PM, Ted Yu <yuzhih...@gmail.com> > > wrote: > > > > > > > > > >> John: > > > > >> In case you want to pursue async poll, it seems (by looking at > > cu

[DISCUSS] KIP-262 Metadata should include the number of state stores for task

Hi all, I would like to discuss a KIP I've submitted : https://cwiki.apache.org/confluence/display/KAFKA/KIP-262%3A+Metadata+should+include+number+of+state+stores+for+task Regards, Richard Yu

Re: [DISCUSS] KIP-262 Metadata should include the number of state stores for task

ng application instances, before the instances > can be restarted with the new code. However, this implies downtime for > an application and is thus not acceptable. > > > -Matthias > > > On 2/24/18 11:11 AM, Richard Yu wrote: > > Hi all, > > > > I would like to

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

After investigation, I have found that the InternalStreamsBuilder#globalTable method is the only instance where the constructor for GlobalKTableImpl is called. The KTableValueGetterSupplier parameter used in this particular constructor is an instance of KTableSourceValueGetterSupplier. Hence, your

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

n details. I will leave comments about that up to the > subsequent PR and to the Kafka Streams folks that are much better suited > than me to comment on them :) > > -Ewen > > On Tue, Jan 2, 2018 at 9:28 PM, Richard Yu <yohan.richard...@gmail.com> > wrote:

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

@Richard: you can close this vote thread with a summary as usual and > update the KIP wiki page accordingly. > > > -Matthias > > On 1/2/18 9:57 PM, Richard Yu wrote: > > A subsequent PR has already been created: > > https://github.com/apache/kafka/pull/

Re: [DISCUSS] KIP-333 Consider a faster form of rebalancing

Hi Becket, I made some changes and clarified the motivation for this KIP. :)It should be easier to understand now since I included a diagram. Thanks,Richard Yu On Tuesday, July 17, 2018, 4:38:11 PM GMT+8, Richard Yu wrote: Hi Becket, Thanks for reviewing this KIP. :) I probably did

Re: [DISCUSS] KIP-333 Consider a faster form of rebalancing

of your questions answered. I will update the KIP soon, so please stay tuned.  Thanks,Richard Yu On Tuesday, July 17, 2018, 2:14:07 PM GMT+8, Becket Qin wrote: Hi Richard, Thanks for the KIP. I am a little confused on what is proposed. The KIP suggests that after recovery from

[DISCUSS] KIP-333 Consider a faster form of rebalancing

Hi all, I would like to discuss KIP-333 (which proposes a faster mode of rebalancing). Here is the link for the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-333%3A+Add+faster+mode+of+rebalancing Thanks, Richard Yu

[DISCUSS] KIP-333 Consider adding faster form of rebalancing

... | | | Thanks,Richard Yu

[DISCUSS] KIP-333 Add faster mode of rebalancing.

... | | | Thanks,Richard Yu

Re: [VOTE] KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde

Nice KIP! +1 (non-binding) -Richard On Friday, July 6, 2018, 9:10:43 AM GMT+8, Matthias J. Sax wrote: Thanks for the KIP! +1 (binding) -Matthias On 7/5/18 7:45 AM, Chia-Ping Tsai wrote: > hi all, > > I would like to start voting on "KIP-331 Add default implementation to >

[DISCUSS] KIP-335 Consider configurations for Kafka Streams

Hi all, Eversince KIP-266 was concluded, there has been a pressing need to migrate Kafka Streams as well. For the link, please click here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-335%3A+Consider+configurations+for+KafkaStreams Thanks, Richard Yu

Re: [DISCUSS] KIP-335 Consider configurations for Kafka Streams

Hi Matthias, It would be nice to get your opinions on this. On Monday, July 9, 2018, 12:17:33 PM GMT+8, Richard Yu wrote: Hi all, Eversince KIP-266 was concluded, there has been a pressing need to migrate Kafka Streams as well. For the link, please click here: https

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

ng calls today that could result in infinite blocking: > > > > Consumer.commitSync() and Consumer.committed(), should they be > > considered > > > > in this KIP as well? > > > > > > > > 3. There are a few other APIs that are today relyin

[VOTE] KIP-266: Add TimeoutException for KafkaConsumer#position

included KafkaConsumer's commitSync, poll, and committed in the KIP. (we will be adding to a TimeoutException to them as well, in a similar manner to what we will be doing for position()) Thanks, Richard Yu

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

Note to all: I have included bounding commitSync() and committed() in this KIP. On Sun, Mar 11, 2018 at 5:05 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Hi all, > > I updated the KIP where overloading position() is now the favored approach. > Bounding position() usin

Re: [DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

Hi John, bq. #1 (wait for metadata) is infinite. Some of what you stated in this KIP has already been previously discussed in a older KIP. (KIP-266) Just for your reference. Thanks, Richard On Tue, Apr 17, 2018 at 11:08 AM, John Roesler wrote: > Hello all, > > I am

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

Hi all, If possible, would a committer please review? Thanks On Sun, Apr 1, 2018 at 7:24 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Hi Guozhang, > > I have clarified the KIP a bit to account for Becket's suggestion on > ClientTimeoutException. > About adding an e

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

to poll(), reading this discussion gave me a new idea for > > providing a non-breaking update path... What if we introduce a new > variant > > 'poll(long timeout, TimeUnit unit)' that displays the new, desired > > behavior, and just leave the old method alone? > > > > T

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

hat is merged, you'll have a clean slate for the rest of the work. > > On Tue, Apr 17, 2018 at 3:39 PM, Richard Yu <yohan.richard...@gmail.com> > wrote: > > > Hi John, > > > > I think that you could finish your PR that corresponds with KIP-288 and > > merge it.

Re: [DISCUSS] KIP-262 Metadata should include the number of state stores for task

he upgrade path is > covered. Thus, you can update your KIP accordingly, referring to KIP-268. > > Can you also update your KIP similar to KIP-268 to cover the old and new > metadata format? > > Thanks! > > -Matthias > > > On 2/24/18 4:07 PM, Richard Yu wrote

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

till add overloading functions and > add a config that will be applied to all overload functions without the > timeout, while for other overloaded functions with the timeout value the > config will be ignored? > > > Guozhang > > On Fri, Mar 30, 2018 at 8:36 PM, Richard Yu &

Re: [VOTE] KIP-274: Kafka Streams Skipped Records Metrics

+1 On Mon, Apr 2, 2018 at 8:42 AM, Guozhang Wang wrote: > +1 (binding). > > On Mon, Apr 2, 2018 at 7:22 AM, Ted Yu wrote: > > > +1 > > > > On Mon, Apr 2, 2018 at 7:11 AM, Bill Bejeck wrote: > > > > > Thanks for the KIP. > > > > > >

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

prevent the tests from hanging? This issue might be out of the KIP, but I prefer it if we could at least make my PR pass the Jenkins Q Thanks On Fri, Mar 30, 2018 at 8:24 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Thanks for the review Becket. > > About the methods b

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

> > > > > > Regarding the flexibility question, has someone tried to dig up the > > > > discussion of the new consumer APIs when they were being written? I > > > vaguely > > > > recall these exact questions about using APIs vs configs and

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

in this KIP? Thanks, Richard On Sat, Mar 17, 2018 at 1:16 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Thanks for the advice, Jason > > I have modified KIP-266 to include the java doc for committed() and other > blocking methods, and I also > mentioned poll() which will

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

Actually, what I said above is inaccurate. In testSeekAndCommitWithBrokerFailures, TestUtils.waitUntilTrue blocks, not seek. My assumption is that seek did not update correctly. I will be digging further into this. On Sat, Mar 17, 2018 at 4:16 PM, Richard Yu <yohan.richard...@gmail.com>

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

. > > Other than that, I think the current KIP seems reasonable. > > Thanks, > Jason > > On Wed, Mar 14, 2018 at 5:00 PM, Richard Yu <yohan.richard...@gmail.com> > wrote: > > > Note to all: I have included bounding commitSync() and committed() in > this >

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

On Mon, Mar 19, 2018 at 6:10 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Hi Ismael, > > You have a great point. Since most of the methods in this KIP have similar > callbacks (position() and committed() both use fetchCommittedOffsets(), > and > commitSync() is simil

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

on() itself will while loop when offsets cannot be retrieved in > the underlying async call. We need to break out this while loop. > 3.3 commitSync() passed Long.MAX_VALUE as the timeout value, we should take > the user specified timeouts when applicable. > > > > Guozhang > &g

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hi Matthias, Thanks for setting up the upgrade path. +1 (non-binding) On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax wrote: > Hi, > > I would like to start the vote for KIP-268: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hi Matthias, Just wondering, once this KIP goes through. Could I restart my older KIP to update SubscriptionInfo? Thanks Richard On Wed, Mar 21, 2018 at 11:18 AM, Matthias J. Sax wrote: > Thanks for following up James. > > > Is this the procedure that happens during

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

; latter in KAFKA-2391, but Jason makes a good point that the overloads are > more flexible. A couple of questions from me: > > 1. Do we need the additional flexibility? > 2. If we do, do we need it for every blocking method? > > Ismael > > On Mon, Mar 19, 2018 at 5:03 PM,

[DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

Hi all, I would like to discuss a potential change which would be made to KafkaConsumer: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75974886 Thanks, Richard Yu

Re: [DISCUSS] KIP-408: Add Asynchronous Processing to Kafka Streams

Hi all, Just want to hear some opinions on this KIP from the PMCs. It would be nice if we got input from them. Don't want to drag this KIP for too long! :) Hope we get some input :) Thanks, Richard On Thu, Jan 3, 2019 at 8:26 PM Richard Yu wrote: > Hi Boyang, > > Interestin

Re: KIP-406: GlobalStreamThread should honor custom reset policy

Hi Matthias, It would be great if we got your input on this. On Sun, Dec 16, 2018 at 3:06 PM Richard Yu wrote: > Hi everybody, > > There is a new KIP regarding the resilience of GlobalStreamThread which > could be seen below: > > https://cwiki.apache.org/confluence/displa

Re: KIP-408: Add Asynchronous Processing to Kafka Streams

e are gonna > provide with this new API, or there is no ordering guarantee at all? Could > we discuss any potential issues if consumer needs to process out-of-order > messages? > > Best, > Boyang > > From: Richard Yu > Sent: Saturday, December 2

Re: KIP-408: Add Asynchronous Processing to Kafka Streams

Hi all, just saying. We are migrating to a different discussion thread. (Forgot that the discussion thread's name was incorrect.) Sorry for the confusion. On Mon, Dec 24, 2018 at 7:57 PM Richard Yu wrote: > Sorry, just making a correction. > > Even if we are processing records out of

Re: [DISCUSS] KIP-408: Add Asynchronous Processing to Kafka Streams

Hi all, Just changing the title of the KIP. Discovered it wasn't right. Thats about it. :) On Mon, Dec 24, 2018 at 7:57 PM Richard Yu wrote: > Sorry, just making a correction. > > Even if we are processing records out of order, we will still have to > checkpoint offset ranges. >

KIP-406: GlobalStreamThread should honor custom reset policy

be great if you could pitch in! Thanks, Richard Yu

Re: KIP-408: Add Asynchronous Processing to Kafka Streams

. Although when implementing this change, there might be some kinks that we have not thought about which could throw a monkey wrench into the works. But definitely worth trying out, Richard On Mon, Dec 24, 2018 at 6:51 PM Richard Yu wrote: > Hi Boyang, > > I could see where you

Re: KIP-408: Add Asynchronous Processing to Kafka Streams

ak > consumer ordering guarantee by default. > > Best, > Boyang > > > From: Richard Yu > Sent: Saturday, December 22, 2018 9:08 AM > To: dev@kafka.apache.org > Subject: Re: KIP-408: Add Asynchronous Processing to Kafka Streams > > Hi Boyang, > > Thanks fo

KIP-408: Add Asynchronous Processing to Kafka Streams

this problem. Thanks, Richard Yu

Re: [DISCUSS] KIP-408: Add Asynchronous Processing to Kafka Streams

d skip records and leave certain records remain on the > queue for late processing. This should be something similar to KIP-408 > which also shares some motivations for us to invest. > > Boyang > > ________ > From: Richard Yu > Sent: Friday, Ja

Re: [DISCUSS] KIP-408: Add Asynchronous Processing to Kafka Streams

Hi all, I made some recent changes to the KIP. It should be more relevant with the issue now (involves Processor API in detail). It would be great if you could comment. Thanks, Richard On Wed, Dec 26, 2018 at 10:01 PM Richard Yu wrote: > Hi all, > > Just changing the title o

Re: [DISCUSS] KIP-408: Add Asynchronous Processing to Kafka Streams

Hi all, Just bumping this KIP. Would be great if we got some discussion. On Sun, Dec 30, 2018 at 5:13 PM Richard Yu wrote: > Hi all, > > I made some recent changes to the KIP. It should be more relevant with the > issue now (involves Processor API in detail). > It would be gre

Re: [DISCUSS] KIP-262 Metadata should include the number of state stores for task

>>> Hi Richard, > >>> > >>> with KIP-268 in place (should be accepted soon) the upgrade path is > >>> covered. Thus, you can update your KIP accordingly, referring to > KIP-268. > >>> > >>> Can you also update your KIP similar to K

KIP-457: Add DISCONNECTED state to Kafka Streams

Hi all, I like to propose a small KIP on adding a new state to KafkaStreams#state(). It is very simple, so this should pass relatively quickly! Here is the discussion link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-457%3A+Add+DISCONNECTED+status+to+Kafka+Streams Cheers, Richard

Re: [VOTE] KIP-457: Add DISCONNECTED status to Kafka Streams

Sorry everybody, if you don't mind holding off voting for a second. Something came up, take a look at the discussion thread. - Richard On Wed, Apr 17, 2019 at 8:46 AM Richard Yu wrote: > Hi all, > > I would like to propose a minor change to the current KafkaStreams#state()

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

would also be dealing with consumer API changes as well? I don't think consumer has any methods which would give us the state of a connection either. - Richard On Wed, Apr 17, 2019 at 8:43 AM Richard Yu wrote: > Hi Micheal, > > Yeah, those are some points I should've clarified. >

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

approach that is now outlined in the KIP. Instead, we could just add a method which I think achieves the same effect. If any of you thinks there is wrong with this approach, please let me know. :) Cheers, Richard On Wed, Apr 17, 2019 at 11:49 AM Richard Yu wrote: > I just realized someth

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

nnect to the brokers. It seems reasonable to > add a DISCONNECT for this case though. > > > > -Matthias > > > > On 4/16/19 9:30 AM, Richard Yu wrote: > > Hi all, > > > > I like to propose a small KIP on adding a new state to > KafkaStreams#state(). > > I

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

Hi all, Considering that this is a simple KIP, I would probably start the voting tomorrow. I think it would be good if we could get this in fast. On Tue, Apr 16, 2019 at 3:31 PM Richard Yu wrote: > Oh, I probably misunderstood the difference between DISCONNECTED and DEAD. > I will

Re: [DISCUSS] KIP-472: Add header to RecordContext

ut would be considered "internal" similar to transaction > markers. However, changing the message format is a mayor change and > hence, I am not sure if it worth doing at all atm. > > > -Matthias > > > > On 5/20/19 7:20 PM, Richard Yu wrote: > > Hello, > >

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

which is not the same as b), but I feel consolidating these to cases with a single metric seem also fine. Guozhang On Wed, Apr 17, 2019 at 2:30 PM Richard Yu wrote: > Alright, so I made a few changes to the KIP. > I realized that there might be an easier way to give the user informat

[DISCUSS] KIP-472: Add header to RecordContext

: https://cwiki.apache.org/confluence/display/KAFKA/KIP-472%3A+%5BSTREAMS%5D+Add+partition+time+field+to+RecordContext Cheers, Richard Yu

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

better user experience. > > Your current proposal does not add a new state, even if it mentions this > in the beginning. Compare: > > https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java#L74-L153 > > > -Matthia

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

Alright, I made some changes. Matthias, if you had time, it would be good if you made another pass. This should be close to completion. Cheers, Richard On Sat, Apr 27, 2019 at 3:46 PM Richard Yu wrote: > Hi Matthias, > > Sure, I could do the DISCONNECTED state. > > > On Sat,

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

as to which approach we should take? It would greatly help in the implementation of the issue. Cheers,Richard On Thursday, June 13, 2019, 4:55:29 PM GMT+8, Richard Yu wrote: Hi Guozhang, Thanks for the input! Then I guess from the approach you have listed above, no API changes will be needed

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

pass if these basic questions aren't > properly sorted out in the KIP. > > Best, > Michael > > > > On Wed, Apr 17, 2019 at 3:35 AM Richard Yu > wrote: > > > Hi all, > > > > Considering that this is a simple KIP, I would probably start the vot

[VOTE] KIP-457: Add DISCONNECTED status to Kafka Streams

Hi all, I would like to propose a minor change to the current KafkaStreams#state() method. Considering the small size of this proposal, I thought it would be good if we could pass it quickly. (It does not have large scale ramifications) Here is the KIP link:

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

Oh, so if possible. I thought it would be good if we could finish this KIP up. Matthias, or Michael, if you have any further comments, please let me know. :) Otherwise, I might restart the voting thread in a few days. Cheers, Richard On Wed, Apr 17, 2019 at 2:30 PM Richard Yu wrote: > Alri

[DISCUSS] KIP-463: Auto-configure serdes passed alongside TopologyBuilder

the pros and cons of each approach. https://cwiki.apache.org/confluence/display/KAFKA/KIP-463%3A+Auto-configure+non-default+Serdes+passed+alongside+the+TopologyBuilder Hope this helps, Richard Yu

[DISCUSS] KIP-515: Reorganize checkpoint system in log cleaner to be per partition

Hi all, A KIP has been written that wishes to upgrade the checkpoint file system in log cleaner. If anybody wishes to comment, feel free to do so. :) https://cwiki.apache.org/confluence/display/KAFKA/KIP-515%3A+Reorganize+checkpoint+file+system+in+log+cleaner+to+be+per+partition Above is the

Re: [DISCUSS] KIP-515: Reorganize checkpoint system in log cleaner to be per partition

mestamp to determine whether the > tombstone should be removed in subsequent rounds of cleaning. This way, we > can still keep the current per disk checkpoint file, which is more > efficient. Personally, I think this approach may be better. Could you > document this approach in the wiki a

Re: [DISCUSS] KIP-515: Reorganize checkpoint system in log cleaner to be per partition

transactions > will be eligible for deletion before later ones. It all depends on the keys > written in the transaction. I don't see an obvious way to solve this > problem without some record-level bookkeeping, but I might be missing > something. > > Thanks, > Jason &g

Re: [DISCUSS] KIP-515: Reorganize checkpoint system in log cleaner to be per partition

ent time + log.cleaner.delete.retention.ms. > > What do you think? > > -Jason > > > > On Thu, Sep 19, 2019 at 3:21 PM Richard Yu > wrote: > > > Hi Jason, > > > > That hadn't occurred to me. > > > > I think I missed your comment in the discus

[DISCUSS] KIP-540: Implement per key stream time tracking

Hi all, After some thought, I thought that we could move forward with this KIP. The low traffic suppression buffer issue is not necessarily a blocker issue, since there are ways to mitigate the problem when implementing this KIP. Below is the KIP link:

Re: [DISCUSS] KIP-539: Implement mechanism to flush out records in low volume suppression buffers

:16 PM Richard Yu wrote: > Hi Bill, > > Thanks for the input! > TBH, I am think that suppression buffers are not used *in response *to > low traffic conditions. > Rather, we are trying to fix the situation when low traffic conditions > occur in a suppression buffer (fo

Re: [VOTE] KIP-534: Retain tombstones for approximately delete.retention.ms milliseconds

he transaction > markers. For proposal 2, one reason is that the interval record header > could be exposed to the clients. > > > Jun > > > On Mon, Oct 14, 2019 at 4:42 PM Richard Yu > wrote: > > > Hi all, > > > > The discussion for KIP-534 seems to have

[VOTE] KIP-534: Retain tombstones for approximately delete.retention.ms milliseconds

Hi all, The discussion for KIP-534 seems to have concluded. So I wish to vote this in so that we can get it done. Its a small bug fix. :) Below is the KIP link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-534%3A+Retain+tombstones+for+approximately+delete.retention.ms+milliseconds

Re: [VOTE] KIP-534: Retain tombstones for approximately delete.retention.ms milliseconds

gt; take more bytes to encode. > > Guozhang > > On Wed, Oct 16, 2019 at 6:48 PM Jason Gustafson > wrote: > > > +1. Thanks Richard. > > > > On Wed, Oct 16, 2019 at 10:04 AM Richard Yu > > wrote: > > > > > Hi all, > > > > >

Re: [VOTE] KIP-534: Retain tombstones for approximately delete.retention.ms milliseconds

Hi all, Seeing that we got all out votes needed with 3 binding votes and 0 nonbinding. I consider this KIP passed. Cheers, Richard On Fri, Oct 18, 2019 at 9:17 AM Guozhang Wang wrote: > Thanks Richard, I'm +1 on the KIP > > On Thu, Oct 17, 2019 at 3:51 PM Richard Yu > wro

  1   2   >