Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-10-14 Thread Yishun Guan
seems the way to go. To address Sophie's concern, in the incoming new mock stores, can we modify the init() to support MockProcessorContext? It sounds like a hack, but I can look into it. Thanks, Yishun On Tue, Oct 8, 2019 at 9:50 PM Yishun Guan wrote: > > Hi, > > I am currently swit

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-10-08 Thread Yishun Guan
at we don't really support using > > state stores in unit testing at all -- you can't > > record the number of expected put/get calls, and you can't use an actual > > store to, well, store things. We definitely > > need both of these things to really round out our unit

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-09-05 Thread Yishun Guan
gt; have them covered within this KIP. We definitely have a gap here and I > think it's pretty clear many users would benefit from state store support > in unit tests! > > Cheers, > Sophie > > On Tue, Aug 27, 2019 at 1:11 PM Yishun Guan wrote: > > > Hi All, > > &g

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-08-27 Thread Yishun Guan
es anyway, ie, this driver would be for advanced users that > build their own stores. > > I think it's actually two orthogonal things and it might even be good to > split both into two KIPs. > > > > -Matthias > > > On 4/30/19 7:52 AM, Yishun Guan wrote: > >

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-30 Thread Yishun Guan
his KIP. > > Does this make sense to you? > > Best, > Bruno > > On Mon, Apr 29, 2019 at 7:11 PM Yishun Guan wrote: > > > Hi, > > > > Sorry for the late reply, I have read through all your valuable > > comments. The KIP still needs work at this point.

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-29 Thread Yishun Guan
de this for the user or to allow them > >>> to > >>> choose between in-memory or rocksDB by setting "persistent" > >>> > >>> 3) I'm wondering if users might want to be able to plug in their own > >>> custom > >>>

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-09 Thread Yishun Guan
Bumping this up again, thanks! On Fri, Apr 5, 2019, 14:36 Yishun Guan wrote: > Hi, bumping this up again. Thanks! > > On Tue, Apr 2, 2019, 13:07 Yishun Guan wrote: > >> Hi All, >> >> I like to start a discussion on KIP-448 >> (https://cwiki.apache.org/conf

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-05 Thread Yishun Guan
Hi, bumping this up again. Thanks! On Tue, Apr 2, 2019, 13:07 Yishun Guan wrote: > Hi All, > > I like to start a discussion on KIP-448 > (https://cwiki.apache.org/confluence/x/SAeZBg). It is about adding > Mock state stores and relevant components for testing purposes. >

[DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-02 Thread Yishun Guan
Hi All, I like to start a discussion on KIP-448 (https://cwiki.apache.org/confluence/x/SAeZBg). It is about adding Mock state stores and relevant components for testing purposes. Here is the JIRA: https://issues.apache.org/jira/browse/KAFKA-6460 This is a rough KIP draft, review and comment are

Questions regarding KAFKA-6460

2019-02-13 Thread Yishun Guan
Hi Guozhang, I have been looking at KAFKA-6460 (https://issues.apache.org/jira/browse/KAFKA-6460?filter=-1) and have a few questions, would you take a look at the comment when you have time? Thank you so much! Best, Yishun

Re: [ANNOUNCE] New Committer: Vahid Hashemian

2019-01-15 Thread Yishun Guan
Congratulations! On Tue, Jan 15, 2019, 16:46 James Cheng wrote: > Congrats, Vahid!! > > -James > > > On Jan 15, 2019, at 2:44 PM, Jason Gustafson wrote: > > > > Hi All, > > > > The PMC for Apache Kafka has invited Vahid Hashemian as a project > committer and > > we are > > pleased to announce

Re: [VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-19 Thread Yishun Guan
> > > On Oct 18, 2018, 5:53 PM -0700, Matthias J. Sax , > wrote: > > +1 (binding) > > > > Thanks for the KIP. And thanks for bumping the thread regularly. As > > 2.1.0 and 2.0.1 releases are running atm, it takes some time to get > > attention. > >

Re: [VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-18 Thread Yishun Guan
Bumping this thread up again, thanks! On Tue, Oct 16, 2018 at 11:24 AM Yishun Guan wrote: > > Bumping this thread up again, thanks! > > On Fri, Oct 12, 2018, 4:53 PM Colin McCabe wrote: >> >> On Fri, Oct 12, 2018, at 15:45, Yishun Guan wrote: >> > Hi Colin, >

Re: [VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-16 Thread Yishun Guan
Bumping this thread up again, thanks! On Fri, Oct 12, 2018, 4:53 PM Colin McCabe wrote: > On Fri, Oct 12, 2018, at 15:45, Yishun Guan wrote: > > Hi Colin, > > > > Thanks for your suggestions. I have modified the current KIP with your > > comments. However, I still thi

Re: [VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-12 Thread Yishun Guan
we could explain why we chose to implement > AutoCloseable rather than Closeable. > > cheers, > Colin > > > On Thu, Oct 11, 2018, at 13:48, Yishun Guan wrote: > > Hi, > > > > Just to bump this voting thread up again. Thanks! > > > > Best, > > Yi

Re: [ANNOUNCE] New Committer: Manikumar Reddy

2018-10-11 Thread Yishun Guan
Congrats Manikumar! On Thu, Oct 11, 2018 at 1:20 PM Sönke Liebau wrote: > > Great news, congratulations Manikumar!! > > On Thu, Oct 11, 2018 at 9:08 PM Vahid Hashemian > wrote: > > > Congrats Manikumar! > > > > On Thu, Oct 11, 2018 at 11:49 AM Ryanne Dolan > > wrote: > > > > > Bravo! > > > > >

Re: [VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-11 Thread Yishun Guan
Hi, Just to bump this voting thread up again. Thanks! Best, Yishun On Fri, Oct 5, 2018 at 12:58 PM Yishun Guan wrote: > > Hi, > > I think we have discussed this well enough to put this into a vote. > > Suggestions are welcome! > > Best, > Yishun > > On Wed,

Re: [DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

2018-10-08 Thread Yishun Guan
; > > > > > Most kafka's classes implementing Closeable/AutoCloseable doesn't > throw checked exception in close() method. Perhaps we should have a > "KafkaCloseable" interface which has a close() method without throwing any > checked exception... > > > >

Re: [VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-05 Thread Yishun Guan
Hi, I think we have discussed this well enough to put this into a vote. Suggestions are welcome! Best, Yishun On Wed, Oct 3, 2018, 2:30 PM Yishun Guan wrote: > Hi All, > > I want to start a voting on this KIP: > https://cwiki.apache.org/confluence/pages/viewpage.action?pag

[VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-03 Thread Yishun Guan
Hi All, I want to start a voting on this KIP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93325308 Here is the discussion thread: https://lists.apache.org/thread.html/9f6394c28d3d11a67600d5d7001e8aaa318f1ad497b50645654bbe3f@%3Cdev.kafka.apache.org%3E Thanks, Yishun

Re: [DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

2018-10-03 Thread Yishun Guan
rdCollector extends AutoCloseable`? > > > > > > -Matthias > > > > > > On 9/27/18 7:46 PM, Chia-Ping Tsai wrote: > > >> (Although I am not quite sure > > >> when one is more desirable than the other) > > > > > > Most

Re: [DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

2018-10-02 Thread Yishun Guan
ot; interface which has a close() method without throwing any > > checked exception... > > > > On 2018/09/27 19:11:20, Yishun Guan wrote: > >> Hi All, > >> > >> Chia-Ping, I agree, similar to VarifiableConsumer, VarifiableProducer > >

Re: [DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

2018-09-27 Thread Yishun Guan
hingLogin > > 5) KafkaChannel > > 6) ConsumerInterceptor > > 7) SelectorMetrics > > 8) HeartbeatThread > > > > Cheers, > > Chia-Ping > > > > > > On 2018/09/26 23:44:31, Yishun Guan wrote: > > > Hi All, > > > > > > Here is a

Re: [DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

2018-09-26 Thread Yishun Guan
better to be explicit with the > proposed changes in a KIP > > Also, you are proposing changes across several components and I'm not sure > if that is possible in a single KIP, but I could very well be wrong on this > one, so we'll see what others say. > > Thanks, >

[DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

2018-09-26 Thread Yishun Guan
Hi All, Here is a trivial KIP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93325308 Suggestions are welcome. Thanks, Yishun

Re: [ANNOUNCE] New committer: Colin McCabe

2018-09-26 Thread Yishun Guan
Congrats! -Yishun On Wed, Sep 26, 2018, 10:04 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > Well deserved! Congratulations Colin. > > -Konstantine > > On Wed, Sep 26, 2018 at 4:57 AM Srinivas Reddy > > wrote: > > > Congratulations Colin  > > > > - > > Srinivas > > > > - Typed

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-09-17 Thread Yishun Guan
@Guozhang Wang What do you think? On Fri, Sep 14, 2018 at 2:39 PM Yishun Guan wrote: > > Hi All, > > After looking into AdminClient.java and ConsumerClient.java, following > the original idea, I think some type specific codes are unavoidable > (we can have a enum class t

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-09-14 Thread Yishun Guan
f them are possibly > > > sending FindCoordinatorRequest (though for consumers today we do not > > > expect > > > it to send for more than one coordinator); personally I think the current > > > approach is better, but I'd like to hear other people's

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-09-06 Thread Yishun Guan
is better, but I'd like to hear other people's opinion as well > > (cc'ed Colin, who implemented the AdminClient). > > > > > > Guozhang > > > > > > On Mon, Sep 3, 2018 at 11:57 AM, Yishun Guan wrote: > > > > > Hi Guozhang, > > > >

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-09-03 Thread Yishun Guan
other requests, which is > also a bit "ugly and dirty", but the change scope may be smaller. General > changes on the AbstractRequestBuilder could be delayed until we realize > this is a common usage for some other requests in their newer versions as > well. > > > Guozhang &g

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-09-02 Thread Yishun Guan
n)` signature, and hence all > requests need to be updated accordingly, but only FindCoordinator for may > return multiple requests in the list, while all others will return > singleton list, right? > > > Guozhang > > > On Fri, Aug 31, 2018 at 10:51 AM, Yishun Guan wrot

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-31 Thread Yishun Guan
@Guozhang Wang Could you review this again when you have time? Thanks! -Yishun On Wed, Aug 29, 2018 at 11:57 AM Yishun Guan wrote: > > Hi, because I have made some significant changes on this design, so I > want to reopen the discussion on this KIP: > https://cwiki.apache.org/conflue

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-29 Thread Yishun Guan
Hi, because I have made some significant changes on this design, so I want to reopen the discussion on this KIP: https://cwiki.apache.org/confluence/x/CgZPBQ Thanks, Yishun On Thu, Aug 16, 2018 at 5:06 PM Yishun Guan wrote: > > I see! Thanks! > > On Thu, Aug 16, 2018, 4:35 PM G

Re: 答复: [VOTE] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-27 Thread Yishun Guan
ist buildFromBatch(). This way will decouple the two > > different build functions. > > > > Then we update the send logic in doSend() correspondingly." > > > > > > You can read about these solutions in more details in this KIP. > > > &

Re: 答复: [VOTE] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-27 Thread Yishun Guan
2018 at 12:12 PM Yishun Guan wrote: > > Thanks for the clarification. I will address this in my KIP. > > On Fri, Aug 17, 2018, 12:06 PM Guozhang Wang wrote: >> >> Today we do have logic for auto down-conversion, but it is assuming a >> one-to-one mapping. The actual

Re: 答复: [VOTE] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-17 Thread Yishun Guan
rsion of the request. > > But current logic do not consider that we may need multiple lower > versioned requests to substitute a single higher versioned request, and > that would be the logic your PR need to address. > > > Guozhang > > On Fri, Aug 17, 2018 at 11:59 AM,

Re: 答复: [VOTE] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-17 Thread Yishun Guan
> that we are going to add. > > > Guozhang > > On Thu, Aug 16, 2018 at 5:46 PM, Hu Xi wrote: > > > +1 (non-binding) > > > > > > 发件人: Yishun Guan > > 发送时间: 2018年8月17日 8:14 > > 收件人: dev@kafka.apache.org >

[VOTE] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-16 Thread Yishun Guan
Hi all, I want to start a vote on this KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-347%3A++Enable+batching+in+FindCoordinatorRequest Here is the discussion thread:

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-16 Thread Yishun Guan
ests with an older > version that only had one group each. > " > > > Guozhang > > > On Wed, Aug 15, 2018 at 4:44 PM, Yishun Guan wrote: > > > Hi, I am looking into AdminClient.scala and AdminClient.java, and also > > looking into ApiVersionRequest.java and

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-15 Thread Yishun Guan
be modified > in the follow-up PR of this KIP. > > > > Guozhang > > On Mon, Aug 13, 2018 at 12:55 PM, Yishun Guan wrote: > > > @Guozhang, thank you so much! > > 1. I agree, fixed. > > 2. Added. > > 3. I see, that is something that I haven't think abo

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-13 Thread Yishun Guan
VersionsRequest in AdminClient, but may need to be extended to handle > one-to-many mapping of different versions). > > This is not sth. that you need to implement under this KIP, but I'd > recommend you think about this earlier than later and see if it may affect > this proposal. &g

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-11 Thread Yishun Guan
dd that later.* > > *Please read your wiki and fix grammatical error such as the following:* > > bq. that need to be make > > Thanks > > On Wed, Aug 8, 2018 at 3:55 PM Yishun Guan wrote: > > > Hi all, > > > > I would like to start a discussion

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-10 Thread Yishun Guan
It would be great if I could get some feedbacks on this KIP, thanks! On Thu, Aug 9, 2018, 10:35 AM Yishun Guan wrote: > To add more context for KIP-347: https://github.com/apache/kafka/pull/5353 > > On Wed, Aug 8, 2018 at 3:55 PM Yishun Guan wrote: > >> Hi all, >>

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-09 Thread Yishun Guan
To add more context for KIP-347: https://github.com/apache/kafka/pull/5353 On Wed, Aug 8, 2018 at 3:55 PM Yishun Guan wrote: > Hi all, > > I would like to start a discussion on: > > KIP-347: Enable batching in FindCoordinatorRequest > https://cwiki.apache.org/confluence/x

[DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

2018-08-08 Thread Yishun Guan
Hi all, I would like to start a discussion on: KIP-347: Enable batching in FindCoordinatorRequest https://cwiki.apache.org/confluence/x/CgZPBQ Thanks @Guozhang Wang for his help and patience! Thanks, Yishun

[jira] [Created] (KAFKA-7206) Enable batching in FindCoordinator

2018-07-25 Thread Yishun Guan (JIRA)
Yishun Guan created KAFKA-7206: -- Summary: Enable batching in FindCoordinator Key: KAFKA-7206 URL: https://issues.apache.org/jira/browse/KAFKA-7206 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-5447) AdminClient metrics

2018-07-14 Thread Yishun Guan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yishun Guan resolved KAFKA-5447. Resolution: Fixed https://cwiki.apache.org/confluence/display/KAFKA/KIP-324%3A+Add+method+to+get

[jira] [Created] (KAFKA-7146) Grouping consumer requests per consumer coordinator in admin client in describeConsumerGroups

2018-07-10 Thread Yishun Guan (JIRA)
Yishun Guan created KAFKA-7146: -- Summary: Grouping consumer requests per consumer coordinator in admin client in describeConsumerGroups Key: KAFKA-7146 URL: https://issues.apache.org/jira/browse/KAFKA-7146

Re: [ANNOUNCE] Apache Kafka 0.11.0.3 Released

2018-07-03 Thread Yishun Guan
Nice! Thanks~ On Tue, Jul 3, 2018, 12:16 PM Ismael Juma wrote: > Thanks Matthias! > > On Tue, 3 Jul 2018, 11:31 Matthias J. Sax, wrote: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA512 > > > > The Apache Kafka community is pleased to announce the release for > > Apache Kafka 0.11.0.3.

Re: [VOTE] KIP-324: Add method to get metrics() in AdminClient

2018-06-29 Thread Yishun Guan
-6788?filter=-1, if so, I can pick this up as my next story. Thanks, Yishun On Fri, Jun 29, 2018 at 12:41 PM, Guozhang Wang wrote: > Yes, please feel free to conclude this thread with a tally. > > > Guozhang > > On Fri, Jun 29, 2018 at 11:53 AM Yishun Guan wrote: > >

Re: [VOTE] KIP-324: Add method to get metrics() in AdminClient

2018-06-29 Thread Yishun Guan
Hi All, Since it has been 72 hrs. Should we go ahead and accept this KIP? Thanks. Best Yishun On Wed, Jun 27, 2018, 4:14 PM Yishun Guan wrote: > I see! Thanks. -Yishun > > On Wed, Jun 27, 2018, 4:10 PM Guozhang Wang wrote: > >> Hi Yishun, >> >> We need to w

Re: [VOTE] KIP-324: Add method to get metrics() in AdminClient

2018-06-27 Thread Yishun Guan
a look and see if they have any > different opinions. > > After 72 hours have passed since you started the vote thread, we can close > it as accepted. > > Guozhang > > On Wed, Jun 27, 2018 at 3:37 PM, Yishun Guan wrote: > > > Added! Thank you Colin. Do we now have

Re: [VOTE] KIP-324: Add method to get metrics() in AdminClient

2018-06-27 Thread Yishun Guan
27, 2018, at 15:23, Colin McCabe wrote: > > On Tue, Jun 26, 2018, at 13:24, Yishun Guan wrote: > > > Hi Colin, > > > > > > I agree with what Guozhang's opinion that because all the other clients > > > have it (producer, consumer..) and this will gain more vi

[VOTE] KIP-324: Add method to get metrics() in AdminClient

2018-06-26 Thread Yishun Guan
Hi All, I am starting a vote on this KIP: https://cwiki.apache.org/confluence/x/lQg0BQ Thanks, Yishun

Re: [VOTE] KIP-324: Add method to get metrics() in AdminClient

2018-06-26 Thread Yishun Guan
t help > other applications that use admin client and want to expose metrics? > > What are the thread-safety guarantees for the map which is returned? > > best, > Colin > > > On Tue, Jun 26, 2018, at 11:29, Yishun Guan wrote: > > Hi All, > > > &

[VOTE] KIP-324: Add method to get metrics() in AdminClient

2018-06-26 Thread Yishun Guan
Hi All, I am starting a vote on this KIP: https://cwiki.apache.org/confluence/x/lQg0BQ Thanks, Yishun

Re: [DISCUSSION] KIP-324: Add method to get metrics() in AdminClient

2018-06-26 Thread Yishun Guan
ients. We can go directly into the voting process. > > Guozhang > > > On Tue, Jun 26, 2018 at 11:15 AM, Yishun Guan wrote: > > > Hi All, > > > > I created a KIP to extract metrics() from AdminClient, here is the link: > > > > https://cwiki.apache.org/con

[DISCUSSION] KIP-324: Add method to get metrics() in AdminClient

2018-06-26 Thread Yishun Guan
Hi All, I created a KIP to extract metrics() from AdminClient, here is the link: https://cwiki.apache.org/confluence/x/lQg0BQ Let me know what you think. Thanks @guozhangwang and @mjsax for guiding me. Best, Yishun

Re: Request access to create KIP

2018-06-25 Thread Yishun Guan
Thank you! On Mon, Jun 25, 2018, 1:03 PM Jason Gustafson wrote: > Done. Thanks for contributing! > > -Jason > > On Mon, Jun 25, 2018 at 12:49 PM, Yishun Guan wrote: > > > Hi, could someone give me access to create KIP? Thanks! - Yishun > > > > On Mon, Jun 2

Re: Request access to create KIP

2018-06-25 Thread Yishun Guan
Hi, could someone give me access to create KIP? Thanks! - Yishun On Mon, Jun 25, 2018, 10:44 AM Yishun Guan wrote: > Hi, my wiki id is gyishun. Thanks! - Yishun >

Request access to create KIP

2018-06-25 Thread Yishun Guan
Hi, my wiki id is gyishun. Thanks! - Yishun