Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-21 Thread Damian Guy
Hi All, There has been one further update to the KIP. WindowedKStream has been renamed to TimeWindowedKStream. Thanks, Damian On Tue, 19 Sep 2017 at 12:21 Damian Guy wrote: > All, a small addition to the KIP. During implementation we realized it > would be good to add the below two methods to

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-19 Thread Damian Guy
All, a small addition to the KIP. During implementation we realized it would be good to add the below two methods to StreamBuilder public synchronized GlobalKTable globalTable(final String topic, final Materialized> materialized) public synchronized KTable table(final String topic, final Materia

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-15 Thread Damian Guy
Sounds good to me. On Thu, 14 Sep 2017 at 19:55 Guozhang Wang wrote: > I'd suggest we remove both to and through together in KIP-182, since for > operator "KTable#to" is as confusing as to "KTable#through" which > overwhelms its benefit as a syntax sugar. I think the extra step "toStream" > is a

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-14 Thread Guozhang Wang
I'd suggest we remove both to and through together in KIP-182, since for operator "KTable#to" is as confusing as to "KTable#through" which overwhelms its benefit as a syntax sugar. I think the extra step "toStream" is actually better to remind the caller that it is sending its changelog stream to t

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-13 Thread Damian Guy
Hi Guozhang, I had an offline discussion with Matthias and Bill about it. It is thought that `to` offers some benefit, i.e., syntactic sugar, so perhaps no harm in keeping it. However, `through` less so, seeing as we can materialize stores via `filter`, `map` etc, so one of the main benefits of `t

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-12 Thread Guozhang Wang
Hi Damian, Why we are deprecating KTable.through while keeping KTable.to? Should we either keep both of them or deprecate both of them in favor or KTable.toStream if people agree that it is confusing to users? Guozhang On Tue, Sep 12, 2017 at 1:18 AM, Damian Guy wrote: > Hi All, > > A minor

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-12 Thread Damian Guy
Hi All, A minor update to the KIP, i needed to add KTable.to(Produced) for consistency. KTable.through will be deprecated in favour of using KTable.toStream().through() Thanks, Damian On Thu, 7 Sep 2017 at 08:52 Damian Guy wrote: > Thanks all. The vote is now closed and the KIP has been accept

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-07 Thread Damian Guy
Thanks all. The vote is now closed and the KIP has been accepted with: 2 non binding votes - bill and matthias 3 binding - Damian, Guozhang, Sriram Regards, Damian On Tue, 5 Sep 2017 at 22:24 Sriram Subramanian wrote: > +1 > > On Tue, Sep 5, 2017 at 1:33 PM, Guozhang Wang wrote: > > > +1 > >

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-05 Thread Sriram Subramanian
+1 On Tue, Sep 5, 2017 at 1:33 PM, Guozhang Wang wrote: > +1 > > On Fri, Sep 1, 2017 at 3:45 PM, Matthias J. Sax > wrote: > > > +1 > > > > On 9/1/17 2:53 PM, Bill Bejeck wrote: > > > +1 > > > > > > On Thu, Aug 31, 2017 at 10:20 AM, Damian Guy > > wrote: > > > > > >> Thanks everyone for voting!

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-05 Thread Guozhang Wang
+1 On Fri, Sep 1, 2017 at 3:45 PM, Matthias J. Sax wrote: > +1 > > On 9/1/17 2:53 PM, Bill Bejeck wrote: > > +1 > > > > On Thu, Aug 31, 2017 at 10:20 AM, Damian Guy > wrote: > > > >> Thanks everyone for voting! Unfortunately i've had to make a bit of an > >> update based on some issues found du

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-01 Thread Matthias J. Sax
+1 On 9/1/17 2:53 PM, Bill Bejeck wrote: > +1 > > On Thu, Aug 31, 2017 at 10:20 AM, Damian Guy wrote: > >> Thanks everyone for voting! Unfortunately i've had to make a bit of an >> update based on some issues found during implementation. >> The main changes are: >> BytesStoreSupplier -> StoreSu

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-01 Thread Bill Bejeck
+1 On Thu, Aug 31, 2017 at 10:20 AM, Damian Guy wrote: > Thanks everyone for voting! Unfortunately i've had to make a bit of an > update based on some issues found during implementation. > The main changes are: > BytesStoreSupplier -> StoreSupplier > Addition of: > WindowBytesStoreSupplier, KeyV

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-31 Thread Damian Guy
Thanks everyone for voting! Unfortunately i've had to make a bit of an update based on some issues found during implementation. The main changes are: BytesStoreSupplier -> StoreSupplier Addition of: WindowBytesStoreSupplier, KeyValueBytesStoreSupplier, SessionBytesStoreSupplier that will restrict s

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Matthias J. Sax
Thanks Damian. Great KIP! +1 -Matthias On 8/25/17 6:45 AM, Damian Guy wrote: > Hi, > > I've just realised we need to add two methods to StateStoreBuilder or it > isn't going to work: > > Map logConfig(); > boolean loggingEnabled(); > > These are needed when we are building the topolo

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Damian Guy
Hi, I've just realised we need to add two methods to StateStoreBuilder or it isn't going to work: Map logConfig(); boolean loggingEnabled(); These are needed when we are building the topology and determining changelog topic names and configs. I've also update the KIP to add StreamBuil

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-24 Thread Sriram Subramanian
+1 On Thu, Aug 24, 2017 at 10:20 AM, Guozhang Wang wrote: > +1. Thanks Damian! > > On Thu, Aug 24, 2017 at 9:47 AM, Bill Bejeck wrote: > > > Thanks for the KIP! > > > > +1 > > > > Thanks, > > Bill > > > > On Thu, Aug 24, 2017 at 12:25 PM, Damian Guy > wrote: > > > > > Hi, > > > > > > I'd like

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-24 Thread Guozhang Wang
+1. Thanks Damian! On Thu, Aug 24, 2017 at 9:47 AM, Bill Bejeck wrote: > Thanks for the KIP! > > +1 > > Thanks, > Bill > > On Thu, Aug 24, 2017 at 12:25 PM, Damian Guy wrote: > > > Hi, > > > > I'd like to kick off the voting thread for KIP-182: > > https://cwiki.apache.org/confluence/display/KA

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-24 Thread Bill Bejeck
Thanks for the KIP! +1 Thanks, Bill On Thu, Aug 24, 2017 at 12:25 PM, Damian Guy wrote: > Hi, > > I'd like to kick off the voting thread for KIP-182: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 182%3A+Reduce+Streams+DSL+overloads+and+allow+easier+ > use+of+custom+storage+engines

[VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-24 Thread Damian Guy
Hi, I'd like to kick off the voting thread for KIP-182: https://cwiki.apache.org/confluence/display/KAFKA/KIP-182%3A+Reduce+Streams+DSL+overloads+and+allow+easier+use+of+custom+storage+engines Thanks, Damian