eams DSL/StateStore Refactoring
Hi Kyle,
Thanks for your input. Really appreciated.
On Thu, 29 Jun 2017 at 06:09 Kyle Winkelman <winkelman.k...@gmail.com>
wrote:
> I like more of a builder pattern even though others have voiced against
> it. The reason I like it is because it makes
I like more of a builder pattern even though others have voiced against it. The
reason I like it is because it makes it clear to the user that a call to
KGroupedStream#count will return a KTable not some intermediate class that I
need to undetstand.
When trying to program in the fluent API
ing votes. I would
> like
> > to
> > > call out for other people especially committers to also take a look at
> > this
> > > proposal and vote.
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Wed, Jun 7, 2017 at 6:37 PM, Kyle Winkelman &
t; > +1 from me
> >
> > any stream should be able to initialize the cogroup
> >
> > On Wed, Jun 14, 2017 at 3:44 PM Kyle Winkelman <winkelman.k...@gmail.com
> >
> > wrote:
> >
> > > I will update the kip to have only the aggregator in the
>> > builder-like pattern. This might make the whole discussion void if we
do
>> > this. Thus, it might make sense to keep this in mind (or even delay
this
>> > KIP?). It seems a waste of time to discuss all this if we are going to
>> > chance it in 2 month anyway...
> API from the current "we are adding more overloads"-pattern to a
> >> > builder-like pattern. This might make the whole discussion void if we
> do
> >> > this. Thus, it might make sense to keep this in mind (or even delay
> this
> >> >
is if we are going to
> > > chance it in 2 month anyway... Just saying.
> > >
> > >
> > > -Matthias
> > >
> > > On 6/13/17 8:05 AM, Michal Borowiecki wrote:
> > > > You're right, I haven't thought of that.
> > > >
> >
nitializer and serde to
>>>
>>> the
>>>
>>> final aggregate statement, since the serde only applies to the state
>>> store,
>>> and the initializer doesn't bear any relation to the first group in
>>> particular. It would end up looking like th
oup(grouped1, aggregator1)
> > >> .cogroup(grouped2, aggregator2)
> > >> .cogroup(grouped3, aggregator3)
> > >> .aggregate(initializer1, aggValueSerde, storeName1);
> > >>
> > >> This doesn't hav
wishList:{}
> > >> > ]
> > >> >
> > >> > 1L, Customer[
> > >> >
> > >> > cart:{Item[no:01]},
> > >> > purchases:{Item[no:07],Item[no:08
tance of whatever the Aggregate class is.
>
> Cheers,
> Damian
>
> On Wed, 31 May 2017 at 20:09 Kyle Winkelman <winkelman.k...@gmail.com>
> wrote:
>
> > Hello all,
> >
> > I have spent some more time on this and the best alternative I have come
> up
>
more sense to me but doesnt allow the user to
specify different initializers for different tables.
Thanks,
Kyle
On May 24, 2017 7:46 PM, "Kyle Winkelman" <winkelman.k...@gmail.com> wrote:
> Yea I really like that idea I'll see what I can do to update the kip and
> my pr
.aggregate(/*per-hour window*/)
> table3 = stream.aggregate(/*per-day window*/)
>
> while underlying we are only using a single store "state-store-name" for
> it.
>
>
> Although this feature is out of the scope of this KIP, I'd like to discuss
> if we can "leave the door open"
> low-level PAPI to achieve this goal.
>
>
> On Fri, May 19, 2017 at 10:04 AM, Kyle Winkelman <winkelman.k...@gmail.com
> > wrote:
>
>> I like your point about not handling other cases such as count and
reduce.
>>
>> I think that reduce may not make se
Hello all,
I would like to start the vote on KIP-150.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-150+-+Kafka-Streams+Cogroup
Thanks,
Kyle
xample, wouldn't the output potentially have more partial results?
I.e, for each input on any stream you'd potentially see a record produced?
I think it is worth mentioning this.
Thanks,
Damian
On Tue, 16 May 2017 at 12:26 Kyle Winkelman <winkelman.k...@gmail.com>
wrote:
> No problem, I
gt; https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.11.0.0 <
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.11.0.0>)
> so people are busier than usual. Stay tuned.
>
> Eno
> > On 15 May 2017, at 13:25, Kyle Winkelman <winkelman.k...@gmail.
Damian Guy, could you let me know if you plan to review this further? There
is no rush, but if you dont have any additional comments I could start the
voting and finish my WIP PR.
Thanks,
Kyle
On May 9, 2017 11:07 AM, "Kyle Winkelman" <winkelman.k...@gmail.com> wrote:
> E
[
https://issues.apache.org/jira/browse/KAFKA-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003248#comment-16003248
]
Kyle Winkelman commented on KAFKA-5206:
---
I have already made a PR but originally named it minor. I
Kyle Winkelman created KAFKA-5206:
-
Summary: RocksDBSessionStore doesn't use default aggSerde.
Key: KAFKA-5206
URL: https://issues.apache.org/jira/browse/KAFKA-5206
Project: Kafka
Issue Type
[
https://issues.apache.org/jira/browse/KAFKA-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003245#comment-16003245
]
Kyle Winkelman commented on KAFKA-5205:
---
I have already made a PR but originally named it minor. I
Kyle Winkelman created KAFKA-5205:
-
Summary: CachingSessionStore doesn't use the default keySerde.
Key: KAFKA-5205
URL: https://issues.apache.org/jira/browse/KAFKA-5205
Project: Kafka
Issue
...@gmail.com> wrote:
> Thanks Kyle, good arguments.
>
> Eno
>
> > On May 7, 2017, at 5:06 PM, Kyle Winkelman <winkelman.k...@gmail.com>
> wrote:
> >
> > *- minor: could you add an exact example (similar to what Jay’s example
> is,
> > or like your
alue, aggregates
> and forwards the result.
> I need to look into it and think about it a bit more, but it seems like it
> could be a useful optimization.
>
> On Thu, 4 May 2017 at 23:21 Kyle Winkelman <winkelman.k...@gmail.com>
> wrote:
>
> > I sure can. I have add
rnal processing to see what store calls
> would be required for both approaches etc.
>
> I think, it's pretty advanced stuff you propose, and it would help to
> understand it better.
>
> Thanks a lot!
>
>
> -Matthias
>
>
>
> On 5/4/17 11:39 AM, Kyle Winkelman wrote:
look at
> > the KIP yet, but will schedule some time to look into it tomorrow. For
> the
> > implementation, can you raise a PR against kafka trunk and mark it as
> WIP?
> > It will be easier to review what you have done.
> >
> > Thanks,
> > Damian
> &
I am replying to this in hopes it will draw some attention to my KIP as I
haven't heard from anyone in a couple days. This is my first KIP and my
first large contribution to the project so I'm sure I did something wrong.
;)
On May 1, 2017 4:18 PM, "Kyle Winkelman" <winkelman.k...@gma
Kyle Winkelman created KAFKA-5172:
-
Summary: CachingSessionStore doesn't fetchPrevious correctly.
Key: KAFKA-5172
URL: https://issues.apache.org/jira/browse/KAFKA-5172
Project: Kafka
Issue
Thanks,
Kyle Winkelman
Hello,
I was hoping to get the ability to edit the wiki so I can post a KIP.
Username: winkelman.kyle
Thanks,
Kyle
it the KIP wiki at
> https://cwiki.apache.org/confluence/display/KAFKA/
> Kafka+Improvement+Proposals. Could you see if you can add this there?
>
> Many thanks
> Eno
>
> On Wed, Apr 26, 2017 at 6:00 PM, Kyle Winkelman <winkelman.k...@gmail.com>
> wrote:
>
>> Thank you
cache before going to the sink.
> Meanwhile another record can come in. So multiple records can be in the
> topology at the same time.
>
> Thanks
> Eno
>
>
>
>
>
> On Fri, Apr 14, 2017 at 8:16 PM, Kyle Winkelman <winkelman.k...@gmail.com>
> wrote:
>
>> En
Eno,
Thanks for the response. The figure was just a restatement of my questions.
I have made an attempt at a low level processor and it appears to work but
it isn't very pretty and was hoping for something at the streams api level.
I have written some code to show an example of how I see the
33 matches
Mail list logo
