Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-10-14 Thread Yishun Guan
Hi, After some investigations, I updated the KIP and the sample MR again: 1. Move the Mocks under test-utils/state 2. extends a WrappedStateStore<>. The reason is that since we decided to provide the testing capability to wrap around a store the developer provides, WrappedStateStore seems the

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-10-08 Thread Yishun Guan
Hi, I am currently switching between jobs - so slightly busier than usual. But I will definitely update this KIP during the weekend. Thanks, Yishun On Mon, Oct 7, 2019 at 3:18 PM Matthias J. Sax wrote: > > What is the status of this KIP? Any updates? > > > -Matthias > > > On 9/10/19 8:08 PM,

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-10-07 Thread Matthias J. Sax
What is the status of this KIP? Any updates? -Matthias On 9/10/19 8:08 PM, Sophie Blee-Goldman wrote: > Just took a look at the current KIP, and I think you should actually be > fine if you're just mocking the stores. > The issue I brought up isn't necessarily blocking this KIP, but it is >

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-09-10 Thread Sophie Blee-Goldman
Just took a look at the current KIP, and I think you should actually be fine if you're just mocking the stores. The issue I brought up isn't necessarily blocking this KIP, but it is related -- just wanted to bring it up and see if there's any overlap, or if it's better to address separately. The

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-09-05 Thread Yishun Guan
Thanks Sophie! I took a look at the issue and the mailing thread. So in other words, people are having issues writing unit tests using in-memory stores (which is a very common practice due to the lack of a better alternative), so we try to provide a better solution for testings, and hopefully

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-08-29 Thread Sophie Blee-Goldman
Hey Yishun! Glad to see this is in the works :) Within the past month or so, needing state stores for unit tests has been brought up multiple times. Unfortunately, before now some people had to rely on internal APIs to get a store for their tests, which is unsafe as they can (and in this case

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-08-27 Thread Yishun Guan
Hi All, I have finally worked on this KIP again and want to discuss with you all before this KIP goes dormant. Recap: https://issues.apache.org/jira/browse/KAFKA-6460 https://cwiki.apache.org/confluence/display/KAFKA/KIP-448%3A+Add+State+Stores+Unit+Test+Support+to+Kafka+Streams+Test+Utils I

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-30 Thread Matthias J. Sax
I just re-read the discussion on the original Jira. It's still a little unclear to me, how this should work end-to-end? It would be good, to describe some test patterns that we want to support first. Maybe using some examples, that show how a test would be written? I don't think that we should

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-30 Thread Yishun Guan
Sounds good! Let me work on this more and add some more information to this KIP before we continue. On Tue, Apr 30, 2019, 00:45 Bruno Cadonna wrote: > Hi Yishun, > > Thank you for continuing with this KIP. IMO, this KIP is very important to > develop robust code. > > I think, a good approach is

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-30 Thread Bruno Cadonna
Hi Yishun, Thank you for continuing with this KIP. IMO, this KIP is very important to develop robust code. I think, a good approach is to do some research on mock development on the internet and in the literatures and then try to prototype the mocks. These activities should yield you a list of

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-29 Thread Yishun Guan
Hi, Sorry for the late reply, I have read through all your valuable comments. The KIP still needs work at this point. I think at this point, one question comes up is that, how should we implement the mock stores - as Sophie suggested, should we open to all Store backend and just wrap around the

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-26 Thread Matthias J. Sax
What is the status of this KIP? Btw: there is also KIP-456. I was wondering if it might be required or helpful to align the design of both with each other. Thoughts? -Matthias On 4/11/19 12:17 AM, Matthias J. Sax wrote: > Thanks for the KIP. Only one initial comment (Sophie mentioned this >

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-11 Thread Matthias J. Sax
Thanks for the KIP. Only one initial comment (Sophie mentioned this already but I want to emphasize on it). You state that > These will be internal classes, so no public API/interface. If this is the case, we don't need a KIP. However, the idea of the original Jira is to actually make those

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-09 Thread Bruno Cadonna
Hi Yishun, Thank you for the KIP. I have a couple of comments: 1. Could you please add an example to the KIP that demonstrates how the mocks should be used in a test? 2. I am wondering, whether the MockKeyValueStore needs to be backed by an actual KeyValueStore (in your KIP

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-09 Thread Sophie Blee-Goldman
Hi Yishun, thanks for the KIP! I have a few initial questions/comments: 1) It may be useful to capture the iterator results as well (eg with a MockIterator that wraps the underlying iterator and records the same way the MockStore wraps/records the underlying store) 2) a. Where is the

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-09 Thread Yishun Guan
Bumping this up again, thanks! On Fri, Apr 5, 2019, 14:36 Yishun Guan wrote: > Hi, bumping this up again. Thanks! > > On Tue, Apr 2, 2019, 13:07 Yishun Guan wrote: > >> Hi All, >> >> I like to start a discussion on KIP-448 >> (https://cwiki.apache.org/confluence/x/SAeZBg). It is about adding

Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-05 Thread Yishun Guan
Hi, bumping this up again. Thanks! On Tue, Apr 2, 2019, 13:07 Yishun Guan wrote: > Hi All, > > I like to start a discussion on KIP-448 > (https://cwiki.apache.org/confluence/x/SAeZBg). It is about adding > Mock state stores and relevant components for testing purposes. > > Here is the JIRA:

[DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

2019-04-02 Thread Yishun Guan
Hi All, I like to start a discussion on KIP-448 (https://cwiki.apache.org/confluence/x/SAeZBg). It is about adding Mock state stores and relevant components for testing purposes. Here is the JIRA: https://issues.apache.org/jira/browse/KAFKA-6460 This is a rough KIP draft, review and comment are