Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Damian Guy]

Thanks +1 (binding)

On Sun, 16 Sep 2018 at 19:37 Nikolay Izhikov  wrote:

> Dear commiters,
>
> I got two binding +1 in [VOTE] thread for this KIP [1].
> I still need one more.
>
> Please, take a look at KIP.
>
> [1]
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
>
> В Чт, 13/09/2018 в 19:33 +0300, Nikolay Izhikov пишет:
> > Fixed.
> >
> > Thanks, for help!
> >
> > Please, take a look and vote.
> >
> > В Чт, 13/09/2018 в 08:40 -0700, Matthias J. Sax пишет:
> > > No need to start a new voting thread :)
> > >
> > > For the KIP update, I think it should be:
> > >
> > > > ReadOnlyWindowStore {
> > > > //Deprecated methods.
> > > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetch(K from, K to, long
> timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetchAll(long timeFrom, long
> timeTo);
> > > >
> > > > //New methods.
> > > > WindowStoreIterator fetch(K key, Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetch(K from, K to, Instant
> from, Duration duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetchAll(Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > }
> > > >
> > > >
> > > > WindowStore {
> > > > //New methods.
> > > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetch(K from, K to, long
> timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetchAll(long timeFrom, long
> timeTo);
> > > > }
> > >
> > > Ie, long-versions are replaced with Instant/Duration in
> > > `ReadOnlyWindowStore`, and `long` method are added in `WindowStore` --
> > > this way, we effectively "move" the long-versions from
> > > `ReadOnlyWindowStore` to `WindowStore`.
> > >
> > > -Matthias
> > >
> > > On 9/13/18 8:08 AM, Nikolay Izhikov wrote:
> > > > Hello, Matthias.
> > > >
> > > > > I like the KIP as-is. Feel free to start a VOTE thread.
> > > >
> > > >
> > > > I'm already started one [1].
> > > > Can you vote in it or I should create a new one?
> > > >
> > > >
> > > > I've updated KIP.
> > > > This has been changed:
> > > >
> > > > ReadOnlyWindowStore {
> > > > //Deprecated methods.
> > > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetch(K from, K to, long
> timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetchAll(long timeFrom, long
> timeTo);
> > > > }
> > > >
> > > > WindowStore {
> > > >   //New methods.
> > > > WindowStoreIterator fetch(K key, Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetch(K from, K to, Instant
> from, Duration duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetchAll(Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > }
> > > >
> > > > [1]
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > > >
> > > > В Ср, 12/09/2018 в 15:46 -0700, Matthias J. Sax пишет:
> > > > > Great!
> > > > >
> > > > > I did not double check the ReadOnlySessionStore interface before,
> and
> > > > > just assumed it would take a timestamp, too. My bad.
> > > > >
> > > > > Please update the KIP for ReadOnlyWindowStore and WindowStore.
> > > > >
> > > > > I like the KIP as-is. Feel free to start a VOTE thread. Even if
> there
> > > > > might be minor follow up comments, we can vote in parallel.
> > > > >
> > > > >
> > > > > -Matthias
> > > > >
> > > > > On 9/12/18 1:06 PM, John Roesler wrote:
> > > > > > Hi Nikolay,
> > > > > >
> > > > > > Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore
> should be
> > > > > > in this KIP.
> > > > > >
> > > > > > And you're right, it seems like ReadOnlySessionStore is not
> necessary to
> > > > > > touch, since it doesn't reference any `long` timestamps.
> > > > > >
> > > > > > Thanks,
> > > > > > -John
> > > > > >
> > > > > > On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov <
> nizhi...@apache.org> wrote:
> > > > > >
> > > > > > > Hello, Matthias.
> > > > > > >
> > > > > > > > His proposal is, to deprecate existing methods on
> `ReadOnlyWindowStore`>
> > > > > > >
> > > > > > > and `ReadOnlySessionStore` and add them to `WindowStore` and>
> `SessionStore`
> > > > > > > > Does this make sense?
> > > > > > >
> > > > > > > You both are experienced Kafka developers, so yes, it does
> make a sense to
> > > > > > > me :).
> > > > > > > Do we want to make this change in KIP-358 or it required
> another KIP?
> > > > > > >
> > > > > > > > Btw: the KIP misses `ReadOnlySessionStore` atm.
> > > > > > >
> > > > > > > Sorry, but I don't understand you.
> > > > > > > As far as I can see, there is only 2 methods in
> `ReadOnlySessionStore`.
> > > > > > > Which method should be migrated to Duration?
> > > > > > >
> > > > > > >
> > > > 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Dear commiters, 

I got two binding +1 in [VOTE] thread for this KIP [1].
I still need one more.

Please, take a look at KIP.

[1] 
https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E

В Чт, 13/09/2018 в 19:33 +0300, Nikolay Izhikov пишет:
> Fixed. 
> 
> Thanks, for help!
> 
> Please, take a look and vote.
> 
> В Чт, 13/09/2018 в 08:40 -0700, Matthias J. Sax пишет:
> > No need to start a new voting thread :)
> > 
> > For the KIP update, I think it should be:
> > 
> > > ReadOnlyWindowStore {
> > > //Deprecated methods.
> > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > KeyValueIterator, V> fetch(K from, K to, long timeFrom, 
> > > long timeTo);
> > > KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> > >  
> > > //New methods.
> > > WindowStoreIterator fetch(K key, Instant from, Duration duration) 
> > > throws IllegalArgumentException;
> > > KeyValueIterator, V> fetch(K from, K to, Instant from, 
> > > Duration duration) throws IllegalArgumentException;
> > > KeyValueIterator, V> fetchAll(Instant from, Duration 
> > > duration) throws IllegalArgumentException;
> > > }
> > >  
> > >  
> > > WindowStore {
> > > //New methods.
> > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > KeyValueIterator, V> fetch(K from, K to, long timeFrom, 
> > > long timeTo);
> > > KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> > > }
> > 
> > Ie, long-versions are replaced with Instant/Duration in
> > `ReadOnlyWindowStore`, and `long` method are added in `WindowStore` --
> > this way, we effectively "move" the long-versions from
> > `ReadOnlyWindowStore` to `WindowStore`.
> > 
> > -Matthias
> > 
> > On 9/13/18 8:08 AM, Nikolay Izhikov wrote:
> > > Hello, Matthias.
> > > 
> > > > I like the KIP as-is. Feel free to start a VOTE thread.
> > > 
> > > 
> > > I'm already started one [1].
> > > Can you vote in it or I should create a new one?
> > > 
> > > 
> > > I've updated KIP.
> > > This has been changed:
> > > 
> > > ReadOnlyWindowStore {
> > > //Deprecated methods.
> > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > KeyValueIterator, V> fetch(K from, K to, long timeFrom, 
> > > long timeTo);
> > > KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> > > }
> > > 
> > > WindowStore {
> > >   //New methods.
> > > WindowStoreIterator fetch(K key, Instant from, Duration duration) 
> > > throws IllegalArgumentException;
> > > KeyValueIterator, V> fetch(K from, K to, Instant from, 
> > > Duration duration) throws IllegalArgumentException;
> > > KeyValueIterator, V> fetchAll(Instant from, Duration 
> > > duration) throws IllegalArgumentException;
> > > }
> > > 
> > > [1] 
> > > https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > > 
> > > В Ср, 12/09/2018 в 15:46 -0700, Matthias J. Sax пишет:
> > > > Great!
> > > > 
> > > > I did not double check the ReadOnlySessionStore interface before, and
> > > > just assumed it would take a timestamp, too. My bad.
> > > > 
> > > > Please update the KIP for ReadOnlyWindowStore and WindowStore.
> > > > 
> > > > I like the KIP as-is. Feel free to start a VOTE thread. Even if there
> > > > might be minor follow up comments, we can vote in parallel.
> > > > 
> > > > 
> > > > -Matthias
> > > > 
> > > > On 9/12/18 1:06 PM, John Roesler wrote:
> > > > > Hi Nikolay,
> > > > > 
> > > > > Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore 
> > > > > should be
> > > > > in this KIP.
> > > > > 
> > > > > And you're right, it seems like ReadOnlySessionStore is not necessary 
> > > > > to
> > > > > touch, since it doesn't reference any `long` timestamps.
> > > > > 
> > > > > Thanks,
> > > > > -John
> > > > > 
> > > > > On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov  
> > > > > wrote:
> > > > > 
> > > > > > Hello, Matthias.
> > > > > > 
> > > > > > > His proposal is, to deprecate existing methods on 
> > > > > > > `ReadOnlyWindowStore`>
> > > > > > 
> > > > > > and `ReadOnlySessionStore` and add them to `WindowStore` and> 
> > > > > > `SessionStore`
> > > > > > > Does this make sense?
> > > > > > 
> > > > > > You both are experienced Kafka developers, so yes, it does make a 
> > > > > > sense to
> > > > > > me :).
> > > > > > Do we want to make this change in KIP-358 or it required another 
> > > > > > KIP?
> > > > > > 
> > > > > > > Btw: the KIP misses `ReadOnlySessionStore` atm.
> > > > > > 
> > > > > > Sorry, but I don't understand you.
> > > > > > As far as I can see, there is only 2 methods in 
> > > > > > `ReadOnlySessionStore`.
> > > > > > Which method should be migrated to Duration?
> > > > > > 
> > > > > > 
> > > > > > https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/ReadOnlySessionStore.java
> > > > > > 
> > > > > > В Вт, 11/09/2018 в 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Fixed. 

Thanks, for help!

Please, take a look and vote.

В Чт, 13/09/2018 в 08:40 -0700, Matthias J. Sax пишет:
> No need to start a new voting thread :)
> 
> For the KIP update, I think it should be:
> 
> > ReadOnlyWindowStore {
> > //Deprecated methods.
> > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > KeyValueIterator, V> fetch(K from, K to, long timeFrom, 
> > long timeTo);
> > KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> >  
> > //New methods.
> > WindowStoreIterator fetch(K key, Instant from, Duration duration) 
> > throws IllegalArgumentException;
> > KeyValueIterator, V> fetch(K from, K to, Instant from, 
> > Duration duration) throws IllegalArgumentException;
> > KeyValueIterator, V> fetchAll(Instant from, Duration 
> > duration) throws IllegalArgumentException;
> > }
> >  
> >  
> > WindowStore {
> > //New methods.
> > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > KeyValueIterator, V> fetch(K from, K to, long timeFrom, 
> > long timeTo);
> > KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> > }
> 
> Ie, long-versions are replaced with Instant/Duration in
> `ReadOnlyWindowStore`, and `long` method are added in `WindowStore` --
> this way, we effectively "move" the long-versions from
> `ReadOnlyWindowStore` to `WindowStore`.
> 
> -Matthias
> 
> On 9/13/18 8:08 AM, Nikolay Izhikov wrote:
> > Hello, Matthias.
> > 
> > > I like the KIP as-is. Feel free to start a VOTE thread.
> > 
> > 
> > I'm already started one [1].
> > Can you vote in it or I should create a new one?
> > 
> > 
> > I've updated KIP.
> > This has been changed:
> > 
> > ReadOnlyWindowStore {
> > //Deprecated methods.
> > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > KeyValueIterator, V> fetch(K from, K to, long timeFrom, 
> > long timeTo);
> > KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> > }
> > 
> > WindowStore {
> > //New methods.
> > WindowStoreIterator fetch(K key, Instant from, Duration duration) 
> > throws IllegalArgumentException;
> > KeyValueIterator, V> fetch(K from, K to, Instant from, 
> > Duration duration) throws IllegalArgumentException;
> > KeyValueIterator, V> fetchAll(Instant from, Duration 
> > duration) throws IllegalArgumentException;
> > }
> > 
> > [1] 
> > https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > 
> > В Ср, 12/09/2018 в 15:46 -0700, Matthias J. Sax пишет:
> > > Great!
> > > 
> > > I did not double check the ReadOnlySessionStore interface before, and
> > > just assumed it would take a timestamp, too. My bad.
> > > 
> > > Please update the KIP for ReadOnlyWindowStore and WindowStore.
> > > 
> > > I like the KIP as-is. Feel free to start a VOTE thread. Even if there
> > > might be minor follow up comments, we can vote in parallel.
> > > 
> > > 
> > > -Matthias
> > > 
> > > On 9/12/18 1:06 PM, John Roesler wrote:
> > > > Hi Nikolay,
> > > > 
> > > > Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore should 
> > > > be
> > > > in this KIP.
> > > > 
> > > > And you're right, it seems like ReadOnlySessionStore is not necessary to
> > > > touch, since it doesn't reference any `long` timestamps.
> > > > 
> > > > Thanks,
> > > > -John
> > > > 
> > > > On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov  
> > > > wrote:
> > > > 
> > > > > Hello, Matthias.
> > > > > 
> > > > > > His proposal is, to deprecate existing methods on 
> > > > > > `ReadOnlyWindowStore`>
> > > > > 
> > > > > and `ReadOnlySessionStore` and add them to `WindowStore` and> 
> > > > > `SessionStore`
> > > > > > Does this make sense?
> > > > > 
> > > > > You both are experienced Kafka developers, so yes, it does make a 
> > > > > sense to
> > > > > me :).
> > > > > Do we want to make this change in KIP-358 or it required another KIP?
> > > > > 
> > > > > > Btw: the KIP misses `ReadOnlySessionStore` atm.
> > > > > 
> > > > > Sorry, but I don't understand you.
> > > > > As far as I can see, there is only 2 methods in 
> > > > > `ReadOnlySessionStore`.
> > > > > Which method should be migrated to Duration?
> > > > > 
> > > > > 
> > > > > https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/ReadOnlySessionStore.java
> > > > > 
> > > > > В Вт, 11/09/2018 в 09:21 -0700, Matthias J. Sax пишет:
> > > > > > I talked to John offline about his last suggestions, that I 
> > > > > > originally
> > > > > > did not fully understand.
> > > > > > 
> > > > > > His proposal is, to deprecate existing methods on 
> > > > > > `ReadOnlyWindowStore`
> > > > > > and `ReadOnlySessionStore` and add them to `WindowStore` and
> > > > > > `SessionStore` (note, all singular -- not to be confused with 
> > > > > > classes
> > > > > > names plural).
> > > > > > 
> > > > > > Btw: the KIP misses `ReadOnlySessionStore` atm.
> > > > > > 
> > > > > > The argument is, that the 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Matthias J. Sax]

No need to start a new voting thread :)

For the KIP update, I think it should be:

> ReadOnlyWindowStore {
> //Deprecated methods.
> WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> KeyValueIterator, V> fetch(K from, K to, long timeFrom, long 
> timeTo);
> KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
>  
> //New methods.
> WindowStoreIterator fetch(K key, Instant from, Duration duration) 
> throws IllegalArgumentException;
> KeyValueIterator, V> fetch(K from, K to, Instant from, 
> Duration duration) throws IllegalArgumentException;
> KeyValueIterator, V> fetchAll(Instant from, Duration 
> duration) throws IllegalArgumentException;
> }
>  
>  
> WindowStore {
> //New methods.
> WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> KeyValueIterator, V> fetch(K from, K to, long timeFrom, long 
> timeTo);
> KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> }

Ie, long-versions are replaced with Instant/Duration in
`ReadOnlyWindowStore`, and `long` method are added in `WindowStore` --
this way, we effectively "move" the long-versions from
`ReadOnlyWindowStore` to `WindowStore`.

-Matthias

On 9/13/18 8:08 AM, Nikolay Izhikov wrote:
> Hello, Matthias.
> 
>> I like the KIP as-is. Feel free to start a VOTE thread.
> 
> 
> I'm already started one [1].
> Can you vote in it or I should create a new one?
> 
> 
> I've updated KIP.
> This has been changed:
> 
> ReadOnlyWindowStore {
> //Deprecated methods.
> WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> KeyValueIterator, V> fetch(K from, K to, long timeFrom, long 
> timeTo);
> KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
> }
> 
> WindowStore {
>   //New methods.
> WindowStoreIterator fetch(K key, Instant from, Duration duration) 
> throws IllegalArgumentException;
> KeyValueIterator, V> fetch(K from, K to, Instant from, 
> Duration duration) throws IllegalArgumentException;
> KeyValueIterator, V> fetchAll(Instant from, Duration 
> duration) throws IllegalArgumentException;
> }
> 
> [1] 
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> 
> В Ср, 12/09/2018 в 15:46 -0700, Matthias J. Sax пишет:
>> Great!
>>
>> I did not double check the ReadOnlySessionStore interface before, and
>> just assumed it would take a timestamp, too. My bad.
>>
>> Please update the KIP for ReadOnlyWindowStore and WindowStore.
>>
>> I like the KIP as-is. Feel free to start a VOTE thread. Even if there
>> might be minor follow up comments, we can vote in parallel.
>>
>>
>> -Matthias
>>
>> On 9/12/18 1:06 PM, John Roesler wrote:
>>> Hi Nikolay,
>>>
>>> Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore should be
>>> in this KIP.
>>>
>>> And you're right, it seems like ReadOnlySessionStore is not necessary to
>>> touch, since it doesn't reference any `long` timestamps.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov  wrote:
>>>
 Hello, Matthias.

> His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`>

 and `ReadOnlySessionStore` and add them to `WindowStore` and> 
 `SessionStore`
> Does this make sense?

 You both are experienced Kafka developers, so yes, it does make a sense to
 me :).
 Do we want to make this change in KIP-358 or it required another KIP?

> Btw: the KIP misses `ReadOnlySessionStore` atm.

 Sorry, but I don't understand you.
 As far as I can see, there is only 2 methods in `ReadOnlySessionStore`.
 Which method should be migrated to Duration?


 https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/ReadOnlySessionStore.java

 В Вт, 11/09/2018 в 09:21 -0700, Matthias J. Sax пишет:
> I talked to John offline about his last suggestions, that I originally
> did not fully understand.
>
> His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`
> and `ReadOnlySessionStore` and add them to `WindowStore` and
> `SessionStore` (note, all singular -- not to be confused with classes
> names plural).
>
> Btw: the KIP misses `ReadOnlySessionStore` atm.
>
> The argument is, that the `ReadOnlyXxxStore` interfaces are only exposed
> via Interactive Queries feature and for this part, using `long` is
> undesired. However, for a `Processor` that reads/writes stores on the
> hot code path, we would like to avoid the object creation overhead and
> stay with `long`. Note, that a `Processor` would use the "read-write"
> interfaces and thus, we can add the more efficient read methods using
> `long` there.
>
> Does this make sense?
>
>
> -Matthias
>
> On 9/11/18 12:20 AM, Nikolay Izhikov wrote:
>> Hello, Guozhang, Bill.
>>
>>> 1) I'd suggest keeping 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Matthias.

> I like the KIP as-is. Feel free to start a VOTE thread.


I'm already started one [1].
Can you vote in it or I should create a new one?


I've updated KIP.
This has been changed:

ReadOnlyWindowStore {
//Deprecated methods.
WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
KeyValueIterator, V> fetch(K from, K to, long timeFrom, long 
timeTo);
KeyValueIterator, V> fetchAll(long timeFrom, long timeTo);
}

WindowStore {
//New methods.
WindowStoreIterator fetch(K key, Instant from, Duration duration) throws 
IllegalArgumentException;
KeyValueIterator, V> fetch(K from, K to, Instant from, Duration 
duration) throws IllegalArgumentException;
KeyValueIterator, V> fetchAll(Instant from, Duration duration) 
throws IllegalArgumentException;
}

[1] 
https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E

В Ср, 12/09/2018 в 15:46 -0700, Matthias J. Sax пишет:
> Great!
> 
> I did not double check the ReadOnlySessionStore interface before, and
> just assumed it would take a timestamp, too. My bad.
> 
> Please update the KIP for ReadOnlyWindowStore and WindowStore.
> 
> I like the KIP as-is. Feel free to start a VOTE thread. Even if there
> might be minor follow up comments, we can vote in parallel.
> 
> 
> -Matthias
> 
> On 9/12/18 1:06 PM, John Roesler wrote:
> > Hi Nikolay,
> > 
> > Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore should be
> > in this KIP.
> > 
> > And you're right, it seems like ReadOnlySessionStore is not necessary to
> > touch, since it doesn't reference any `long` timestamps.
> > 
> > Thanks,
> > -John
> > 
> > On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov  wrote:
> > 
> > > Hello, Matthias.
> > > 
> > > > His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`>
> > > 
> > > and `ReadOnlySessionStore` and add them to `WindowStore` and> 
> > > `SessionStore`
> > > > Does this make sense?
> > > 
> > > You both are experienced Kafka developers, so yes, it does make a sense to
> > > me :).
> > > Do we want to make this change in KIP-358 or it required another KIP?
> > > 
> > > > Btw: the KIP misses `ReadOnlySessionStore` atm.
> > > 
> > > Sorry, but I don't understand you.
> > > As far as I can see, there is only 2 methods in `ReadOnlySessionStore`.
> > > Which method should be migrated to Duration?
> > > 
> > > 
> > > https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/ReadOnlySessionStore.java
> > > 
> > > В Вт, 11/09/2018 в 09:21 -0700, Matthias J. Sax пишет:
> > > > I talked to John offline about his last suggestions, that I originally
> > > > did not fully understand.
> > > > 
> > > > His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`
> > > > and `ReadOnlySessionStore` and add them to `WindowStore` and
> > > > `SessionStore` (note, all singular -- not to be confused with classes
> > > > names plural).
> > > > 
> > > > Btw: the KIP misses `ReadOnlySessionStore` atm.
> > > > 
> > > > The argument is, that the `ReadOnlyXxxStore` interfaces are only exposed
> > > > via Interactive Queries feature and for this part, using `long` is
> > > > undesired. However, for a `Processor` that reads/writes stores on the
> > > > hot code path, we would like to avoid the object creation overhead and
> > > > stay with `long`. Note, that a `Processor` would use the "read-write"
> > > > interfaces and thus, we can add the more efficient read methods using
> > > > `long` there.
> > > > 
> > > > Does this make sense?
> > > > 
> > > > 
> > > > -Matthias
> > > > 
> > > > On 9/11/18 12:20 AM, Nikolay Izhikov wrote:
> > > > > Hello, Guozhang, Bill.
> > > > > 
> > > > > > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is
> > > > > 
> > > > > I am agree with you.
> > > > > Currently, `Punctuator` edits are not included in KIP.
> > > > > 
> > > > > > 2) I'm fine with keeping KeyValueStore extending
> > > 
> > > ReadOnlyKeyValueStore
> > > > > 
> > > > > Great, currently, there is no suggested API change in `KeyValueStore`
> > > 
> > > or `ReadOnlyKeyValueStore`.
> > > > > 
> > > > > Seems, you agree with all KIP details.
> > > > > Can you vote, please? [1]
> > > > > 
> > > > > [1]
> > > 
> > > https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > > > > 
> > > > > 
> > > > > В Пн, 10/09/2018 в 19:49 -0400, Bill Bejeck пишет:
> > > > > > Hi Nikolay,
> > > > > > 
> > > > > > I'm a +1 to points 1 and 2 above from Guozhang.
> > > > > > 
> > > > > > Thanks,
> > > > > > Bill
> > > > > > 
> > > > > > On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang 
> > > 
> > > wrote:
> > > > > > 
> > > > > > > Hello Nikolay,
> > > > > > > 
> > > > > > > Thanks for picking this up! Just sharing my two cents:
> > > > > > > 
> > > > > > > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as
> > > 
> > > is since
> > > > > > > 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Matthias J. Sax]

Great!

I did not double check the ReadOnlySessionStore interface before, and
just assumed it would take a timestamp, too. My bad.

Please update the KIP for ReadOnlyWindowStore and WindowStore.

I like the KIP as-is. Feel free to start a VOTE thread. Even if there
might be minor follow up comments, we can vote in parallel.


-Matthias

On 9/12/18 1:06 PM, John Roesler wrote:
> Hi Nikolay,
> 
> Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore should be
> in this KIP.
> 
> And you're right, it seems like ReadOnlySessionStore is not necessary to
> touch, since it doesn't reference any `long` timestamps.
> 
> Thanks,
> -John
> 
> On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov  wrote:
> 
>> Hello, Matthias.
>>
>>> His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`>
>> and `ReadOnlySessionStore` and add them to `WindowStore` and> `SessionStore`
>>> Does this make sense?
>>
>> You both are experienced Kafka developers, so yes, it does make a sense to
>> me :).
>> Do we want to make this change in KIP-358 or it required another KIP?
>>
>>> Btw: the KIP misses `ReadOnlySessionStore` atm.
>>
>> Sorry, but I don't understand you.
>> As far as I can see, there is only 2 methods in `ReadOnlySessionStore`.
>> Which method should be migrated to Duration?
>>
>>
>> https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/ReadOnlySessionStore.java
>>
>> В Вт, 11/09/2018 в 09:21 -0700, Matthias J. Sax пишет:
>>> I talked to John offline about his last suggestions, that I originally
>>> did not fully understand.
>>>
>>> His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`
>>> and `ReadOnlySessionStore` and add them to `WindowStore` and
>>> `SessionStore` (note, all singular -- not to be confused with classes
>>> names plural).
>>>
>>> Btw: the KIP misses `ReadOnlySessionStore` atm.
>>>
>>> The argument is, that the `ReadOnlyXxxStore` interfaces are only exposed
>>> via Interactive Queries feature and for this part, using `long` is
>>> undesired. However, for a `Processor` that reads/writes stores on the
>>> hot code path, we would like to avoid the object creation overhead and
>>> stay with `long`. Note, that a `Processor` would use the "read-write"
>>> interfaces and thus, we can add the more efficient read methods using
>>> `long` there.
>>>
>>> Does this make sense?
>>>
>>>
>>> -Matthias
>>>
>>> On 9/11/18 12:20 AM, Nikolay Izhikov wrote:
 Hello, Guozhang, Bill.

> 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is

 I am agree with you.
 Currently, `Punctuator` edits are not included in KIP.

> 2) I'm fine with keeping KeyValueStore extending
>> ReadOnlyKeyValueStore

 Great, currently, there is no suggested API change in `KeyValueStore`
>> or `ReadOnlyKeyValueStore`.

 Seems, you agree with all KIP details.
 Can you vote, please? [1]

 [1]
>> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E


 В Пн, 10/09/2018 в 19:49 -0400, Bill Bejeck пишет:
> Hi Nikolay,
>
> I'm a +1 to points 1 and 2 above from Guozhang.
>
> Thanks,
> Bill
>
> On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang 
>> wrote:
>
>> Hello Nikolay,
>>
>> Thanks for picking this up! Just sharing my two cents:
>>
>> 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as
>> is since
>> comparing with other places where we are replacing with Duration
>> and
>> Instant, this is not a user specified value as part of the DSL but
>> rather a
>> passed-in parameter, plus with high punctuation frequency creating
>> a new
>> instance of Instant may be costly.
>>
>> 2) I'm fine with keeping KeyValueStore extending
>> ReadOnlyKeyValueStore with
>> APIs of `long` as well as inheriting APIs of `Duration`.
>>
>>
>> Guozhang
>>
>>
>> On Mon, Sep 10, 2018 at 11:11 AM, Nikolay Izhikov <
>> nizhi...@apache.org>
>> wrote:
>>
>>> Hello, Matthias.
>>>
 (4) While I agree that we might want to deprecate it, I am not
>> sure if
>>>
>>> this should be part of this KIP?
 Seems to be unrelated?
 Should this have been part of KIP-319?
 If yes, we might still want to updated this other KIP? WDYT?
>>>
>>> OK, I removed this deprecation from this KIP.
>>>
>>> Please, tell me, is there anything else that should be improved
>> to make
>>> this KIP ready to be implemented.
>>>
>>> В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
 (1) Sounds good to me, to just use IllegalArgumentException
>> for both --
 and thanks for pointing out that Duration can be negative and
>> we need
>>
>> to
 check for this. For the KIP, it would be nice to add to all
>> methods
>>
>> than
 (even if we don't do it in 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[John Roesler]

Hi Nikolay,

Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore should be
in this KIP.

And you're right, it seems like ReadOnlySessionStore is not necessary to
touch, since it doesn't reference any `long` timestamps.

Thanks,
-John

On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov  wrote:

> Hello, Matthias.
>
> > His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`>
> and `ReadOnlySessionStore` and add them to `WindowStore` and> `SessionStore`
> > Does this make sense?
>
> You both are experienced Kafka developers, so yes, it does make a sense to
> me :).
> Do we want to make this change in KIP-358 or it required another KIP?
>
> > Btw: the KIP misses `ReadOnlySessionStore` atm.
>
> Sorry, but I don't understand you.
> As far as I can see, there is only 2 methods in `ReadOnlySessionStore`.
> Which method should be migrated to Duration?
>
>
> https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/ReadOnlySessionStore.java
>
> В Вт, 11/09/2018 в 09:21 -0700, Matthias J. Sax пишет:
> > I talked to John offline about his last suggestions, that I originally
> > did not fully understand.
> >
> > His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`
> > and `ReadOnlySessionStore` and add them to `WindowStore` and
> > `SessionStore` (note, all singular -- not to be confused with classes
> > names plural).
> >
> > Btw: the KIP misses `ReadOnlySessionStore` atm.
> >
> > The argument is, that the `ReadOnlyXxxStore` interfaces are only exposed
> > via Interactive Queries feature and for this part, using `long` is
> > undesired. However, for a `Processor` that reads/writes stores on the
> > hot code path, we would like to avoid the object creation overhead and
> > stay with `long`. Note, that a `Processor` would use the "read-write"
> > interfaces and thus, we can add the more efficient read methods using
> > `long` there.
> >
> > Does this make sense?
> >
> >
> > -Matthias
> >
> > On 9/11/18 12:20 AM, Nikolay Izhikov wrote:
> > > Hello, Guozhang, Bill.
> > >
> > > > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is
> > >
> > > I am agree with you.
> > > Currently, `Punctuator` edits are not included in KIP.
> > >
> > > > 2) I'm fine with keeping KeyValueStore extending
> ReadOnlyKeyValueStore
> > >
> > > Great, currently, there is no suggested API change in `KeyValueStore`
> or `ReadOnlyKeyValueStore`.
> > >
> > > Seems, you agree with all KIP details.
> > > Can you vote, please? [1]
> > >
> > > [1]
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > >
> > >
> > > В Пн, 10/09/2018 в 19:49 -0400, Bill Bejeck пишет:
> > > > Hi Nikolay,
> > > >
> > > > I'm a +1 to points 1 and 2 above from Guozhang.
> > > >
> > > > Thanks,
> > > > Bill
> > > >
> > > > On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang 
> wrote:
> > > >
> > > > > Hello Nikolay,
> > > > >
> > > > > Thanks for picking this up! Just sharing my two cents:
> > > > >
> > > > > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as
> is since
> > > > > comparing with other places where we are replacing with Duration
> and
> > > > > Instant, this is not a user specified value as part of the DSL but
> rather a
> > > > > passed-in parameter, plus with high punctuation frequency creating
> a new
> > > > > instance of Instant may be costly.
> > > > >
> > > > > 2) I'm fine with keeping KeyValueStore extending
> ReadOnlyKeyValueStore with
> > > > > APIs of `long` as well as inheriting APIs of `Duration`.
> > > > >
> > > > >
> > > > > Guozhang
> > > > >
> > > > >
> > > > > On Mon, Sep 10, 2018 at 11:11 AM, Nikolay Izhikov <
> nizhi...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Hello, Matthias.
> > > > > >
> > > > > > > (4) While I agree that we might want to deprecate it, I am not
> sure if
> > > > > >
> > > > > > this should be part of this KIP?
> > > > > > > Seems to be unrelated?
> > > > > > > Should this have been part of KIP-319?
> > > > > > > If yes, we might still want to updated this other KIP? WDYT?
> > > > > >
> > > > > > OK, I removed this deprecation from this KIP.
> > > > > >
> > > > > > Please, tell me, is there anything else that should be improved
> to make
> > > > > > this KIP ready to be implemented.
> > > > > >
> > > > > > В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
> > > > > > > (1) Sounds good to me, to just use IllegalArgumentException
> for both --
> > > > > > > and thanks for pointing out that Duration can be negative and
> we need
> > > > >
> > > > > to
> > > > > > > check for this. For the KIP, it would be nice to add to all
> methods
> > > > >
> > > > > than
> > > > > > > (even if we don't do it in the code but only document in
> JavaDocs).
> > > > > > >
> > > > > > > (2) I would argue for a new single method interface. Not sure
> about the
> > > > > > > name though.
> > > > > > >
> > > > > > > (3) Even if `#fetch(K, K, long, long)` and 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Matthias.

> His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`> and 
> `ReadOnlySessionStore` and add them to `WindowStore` and> `SessionStore`
> Does this make sense?

You both are experienced Kafka developers, so yes, it does make a sense to me 
:).
Do we want to make this change in KIP-358 or it required another KIP?

> Btw: the KIP misses `ReadOnlySessionStore` atm.

Sorry, but I don't understand you.
As far as I can see, there is only 2 methods in `ReadOnlySessionStore`.
Which method should be migrated to Duration?

https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/ReadOnlySessionStore.java

В Вт, 11/09/2018 в 09:21 -0700, Matthias J. Sax пишет:
> I talked to John offline about his last suggestions, that I originally
> did not fully understand.
> 
> His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`
> and `ReadOnlySessionStore` and add them to `WindowStore` and
> `SessionStore` (note, all singular -- not to be confused with classes
> names plural).
> 
> Btw: the KIP misses `ReadOnlySessionStore` atm.
> 
> The argument is, that the `ReadOnlyXxxStore` interfaces are only exposed
> via Interactive Queries feature and for this part, using `long` is
> undesired. However, for a `Processor` that reads/writes stores on the
> hot code path, we would like to avoid the object creation overhead and
> stay with `long`. Note, that a `Processor` would use the "read-write"
> interfaces and thus, we can add the more efficient read methods using
> `long` there.
> 
> Does this make sense?
> 
> 
> -Matthias
> 
> On 9/11/18 12:20 AM, Nikolay Izhikov wrote:
> > Hello, Guozhang, Bill.
> > 
> > > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is
> > 
> > I am agree with you.
> > Currently, `Punctuator` edits are not included in KIP.
> > 
> > > 2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore
> > 
> > Great, currently, there is no suggested API change in `KeyValueStore` or 
> > `ReadOnlyKeyValueStore`.
> > 
> > Seems, you agree with all KIP details.
> > Can you vote, please? [1]
> > 
> > [1] 
> > https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > 
> > 
> > В Пн, 10/09/2018 в 19:49 -0400, Bill Bejeck пишет:
> > > Hi Nikolay,
> > > 
> > > I'm a +1 to points 1 and 2 above from Guozhang.
> > > 
> > > Thanks,
> > > Bill
> > > 
> > > On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang  wrote:
> > > 
> > > > Hello Nikolay,
> > > > 
> > > > Thanks for picking this up! Just sharing my two cents:
> > > > 
> > > > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is 
> > > > since
> > > > comparing with other places where we are replacing with Duration and
> > > > Instant, this is not a user specified value as part of the DSL but 
> > > > rather a
> > > > passed-in parameter, plus with high punctuation frequency creating a new
> > > > instance of Instant may be costly.
> > > > 
> > > > 2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore 
> > > > with
> > > > APIs of `long` as well as inheriting APIs of `Duration`.
> > > > 
> > > > 
> > > > Guozhang
> > > > 
> > > > 
> > > > On Mon, Sep 10, 2018 at 11:11 AM, Nikolay Izhikov 
> > > > wrote:
> > > > 
> > > > > Hello, Matthias.
> > > > > 
> > > > > > (4) While I agree that we might want to deprecate it, I am not sure 
> > > > > > if
> > > > > 
> > > > > this should be part of this KIP?
> > > > > > Seems to be unrelated?
> > > > > > Should this have been part of KIP-319?
> > > > > > If yes, we might still want to updated this other KIP? WDYT?
> > > > > 
> > > > > OK, I removed this deprecation from this KIP.
> > > > > 
> > > > > Please, tell me, is there anything else that should be improved to 
> > > > > make
> > > > > this KIP ready to be implemented.
> > > > > 
> > > > > В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
> > > > > > (1) Sounds good to me, to just use IllegalArgumentException for 
> > > > > > both --
> > > > > > and thanks for pointing out that Duration can be negative and we 
> > > > > > need
> > > > 
> > > > to
> > > > > > check for this. For the KIP, it would be nice to add to all methods
> > > > 
> > > > than
> > > > > > (even if we don't do it in the code but only document in JavaDocs).
> > > > > > 
> > > > > > (2) I would argue for a new single method interface. Not sure about 
> > > > > > the
> > > > > > name though.
> > > > > > 
> > > > > > (3) Even if `#fetch(K, K, long, long)` and `#fetchAll(long, long)` 
> > > > > > is
> > > > > > _currently_ not used internally, I would still argue they are both 
> > > > > > dual
> > > > > > use -- we might all a new DSL operator at any point that uses those
> > > > > > methods. Thus to be "future prove" I would consider them dual use.
> > > > > > 
> > > > > > > Since the ReadOnlyWindowStore is only used by IQ,
> > > > > > 
> > > > > > This contradicts your other statement:
> > > > > > 
> 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Matthias J. Sax]

I talked to John offline about his last suggestions, that I originally
did not fully understand.

His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`
and `ReadOnlySessionStore` and add them to `WindowStore` and
`SessionStore` (note, all singular -- not to be confused with classes
names plural).

Btw: the KIP misses `ReadOnlySessionStore` atm.

The argument is, that the `ReadOnlyXxxStore` interfaces are only exposed
via Interactive Queries feature and for this part, using `long` is
undesired. However, for a `Processor` that reads/writes stores on the
hot code path, we would like to avoid the object creation overhead and
stay with `long`. Note, that a `Processor` would use the "read-write"
interfaces and thus, we can add the more efficient read methods using
`long` there.

Does this make sense?


-Matthias

On 9/11/18 12:20 AM, Nikolay Izhikov wrote:
> Hello, Guozhang, Bill.
> 
>> 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is
> 
> I am agree with you.
> Currently, `Punctuator` edits are not included in KIP.
> 
>> 2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore
> 
> Great, currently, there is no suggested API change in `KeyValueStore` or 
> `ReadOnlyKeyValueStore`.
> 
> Seems, you agree with all KIP details.
> Can you vote, please? [1]
> 
> [1] 
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> 
> 
> В Пн, 10/09/2018 в 19:49 -0400, Bill Bejeck пишет:
>> Hi Nikolay,
>>
>> I'm a +1 to points 1 and 2 above from Guozhang.
>>
>> Thanks,
>> Bill
>>
>> On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang  wrote:
>>
>>> Hello Nikolay,
>>>
>>> Thanks for picking this up! Just sharing my two cents:
>>>
>>> 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is since
>>> comparing with other places where we are replacing with Duration and
>>> Instant, this is not a user specified value as part of the DSL but rather a
>>> passed-in parameter, plus with high punctuation frequency creating a new
>>> instance of Instant may be costly.
>>>
>>> 2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore with
>>> APIs of `long` as well as inheriting APIs of `Duration`.
>>>
>>>
>>> Guozhang
>>>
>>>
>>> On Mon, Sep 10, 2018 at 11:11 AM, Nikolay Izhikov 
>>> wrote:
>>>
 Hello, Matthias.

> (4) While I agree that we might want to deprecate it, I am not sure if

 this should be part of this KIP?
> Seems to be unrelated?
> Should this have been part of KIP-319?
> If yes, we might still want to updated this other KIP? WDYT?

 OK, I removed this deprecation from this KIP.

 Please, tell me, is there anything else that should be improved to make
 this KIP ready to be implemented.

 В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
> (1) Sounds good to me, to just use IllegalArgumentException for both --
> and thanks for pointing out that Duration can be negative and we need
>>>
>>> to
> check for this. For the KIP, it would be nice to add to all methods
>>>
>>> than
> (even if we don't do it in the code but only document in JavaDocs).
>
> (2) I would argue for a new single method interface. Not sure about the
> name though.
>
> (3) Even if `#fetch(K, K, long, long)` and `#fetchAll(long, long)` is
> _currently_ not used internally, I would still argue they are both dual
> use -- we might all a new DSL operator at any point that uses those
> methods. Thus to be "future prove" I would consider them dual use.
>
>> Since the ReadOnlyWindowStore is only used by IQ,
>
> This contradicts your other statement:
>
>> org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is

 used
>> in KStreamWindowAggregate.
>
> Or do you suggest to move `fetch(K, long)` from `ReadOnlyWindowStore`
>>>
>>> to
> `WindowStore` ? This would not make sense IMHO, as `WindowStore extends
> ReadOnlyWindowStore` and thus, we would loose this method for IQ.
>
>
> (4) While I agree that we might want to deprecate it, I am not sure if
> this should be part of this KIP? Seems to be unrelated? Should this
>>>
>>> have
> been part of KIP-319? If yes, we might still want to updated this other
> KIP? WDYT?
>
>
> -Matthias
>
>
> On 9/7/18 12:09 PM, John Roesler wrote:
>> Hey all,
>>
>> (1): Duration can be negative, just like long. We need to enforce any
>> bounds that we currently enforce. We don't need the `throws`

 declaration
>> for runtime exceptions, but the potential IllegalArgumentException

 should
>> be documented in the javadoc for these methods. I still feel that

 surfacing
>> the ArithmeticException directly would not be a great experience, so
>>>
>>> I
>> still advocate for wrapping it in an IllegalArgumentException that

 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Guozhang, Bill.

> 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is

I am agree with you.
Currently, `Punctuator` edits are not included in KIP.

> 2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore

Great, currently, there is no suggested API change in `KeyValueStore` or 
`ReadOnlyKeyValueStore`.

Seems, you agree with all KIP details.
Can you vote, please? [1]

[1] 
https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E


В Пн, 10/09/2018 в 19:49 -0400, Bill Bejeck пишет:
> Hi Nikolay,
> 
> I'm a +1 to points 1 and 2 above from Guozhang.
> 
> Thanks,
> Bill
> 
> On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang  wrote:
> 
> > Hello Nikolay,
> > 
> > Thanks for picking this up! Just sharing my two cents:
> > 
> > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is since
> > comparing with other places where we are replacing with Duration and
> > Instant, this is not a user specified value as part of the DSL but rather a
> > passed-in parameter, plus with high punctuation frequency creating a new
> > instance of Instant may be costly.
> > 
> > 2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore with
> > APIs of `long` as well as inheriting APIs of `Duration`.
> > 
> > 
> > Guozhang
> > 
> > 
> > On Mon, Sep 10, 2018 at 11:11 AM, Nikolay Izhikov 
> > wrote:
> > 
> > > Hello, Matthias.
> > > 
> > > > (4) While I agree that we might want to deprecate it, I am not sure if
> > > 
> > > this should be part of this KIP?
> > > > Seems to be unrelated?
> > > > Should this have been part of KIP-319?
> > > > If yes, we might still want to updated this other KIP? WDYT?
> > > 
> > > OK, I removed this deprecation from this KIP.
> > > 
> > > Please, tell me, is there anything else that should be improved to make
> > > this KIP ready to be implemented.
> > > 
> > > В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
> > > > (1) Sounds good to me, to just use IllegalArgumentException for both --
> > > > and thanks for pointing out that Duration can be negative and we need
> > 
> > to
> > > > check for this. For the KIP, it would be nice to add to all methods
> > 
> > than
> > > > (even if we don't do it in the code but only document in JavaDocs).
> > > > 
> > > > (2) I would argue for a new single method interface. Not sure about the
> > > > name though.
> > > > 
> > > > (3) Even if `#fetch(K, K, long, long)` and `#fetchAll(long, long)` is
> > > > _currently_ not used internally, I would still argue they are both dual
> > > > use -- we might all a new DSL operator at any point that uses those
> > > > methods. Thus to be "future prove" I would consider them dual use.
> > > > 
> > > > > Since the ReadOnlyWindowStore is only used by IQ,
> > > > 
> > > > This contradicts your other statement:
> > > > 
> > > > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is
> > > 
> > > used
> > > > > in KStreamWindowAggregate.
> > > > 
> > > > Or do you suggest to move `fetch(K, long)` from `ReadOnlyWindowStore`
> > 
> > to
> > > > `WindowStore` ? This would not make sense IMHO, as `WindowStore extends
> > > > ReadOnlyWindowStore` and thus, we would loose this method for IQ.
> > > > 
> > > > 
> > > > (4) While I agree that we might want to deprecate it, I am not sure if
> > > > this should be part of this KIP? Seems to be unrelated? Should this
> > 
> > have
> > > > been part of KIP-319? If yes, we might still want to updated this other
> > > > KIP? WDYT?
> > > > 
> > > > 
> > > > -Matthias
> > > > 
> > > > 
> > > > On 9/7/18 12:09 PM, John Roesler wrote:
> > > > > Hey all,
> > > > > 
> > > > > (1): Duration can be negative, just like long. We need to enforce any
> > > > > bounds that we currently enforce. We don't need the `throws`
> > > 
> > > declaration
> > > > > for runtime exceptions, but the potential IllegalArgumentException
> > > 
> > > should
> > > > > be documented in the javadoc for these methods. I still feel that
> > > 
> > > surfacing
> > > > > the ArithmeticException directly would not be a great experience, so
> > 
> > I
> > > > > still advocate for wrapping it in an IllegalArgumentException that
> > > 
> > > explains
> > > > > our upper bound for Duration is "max-long number of milliseconds"
> > > > > 
> > > > > (2): I agree with your performance intuition. I don't think creating
> > > 
> > > one
> > > > > object per call to punctuate is going to substantially affect the
> > > > > performance.
> > > > > 
> > > > > I think the deeper problem with Punctuator is that it's currently a
> > 
> > SAM
> > > > > interface. If we add a new method to it, we break the source code of
> > > 
> > > anyone
> > > > > passing a function. We can add the new method with a default
> > > > > implementation, as Nikolay suggested, but then you get into figuring
> > > 
> > > out
> > > > > which one to default, and no one's happy. Alternatively, we can just
> > > 
> > > make a
> > > 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Bill Bejeck]

Hi Nikolay,

I'm a +1 to points 1 and 2 above from Guozhang.

Thanks,
Bill

On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang  wrote:

> Hello Nikolay,
>
> Thanks for picking this up! Just sharing my two cents:
>
> 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is since
> comparing with other places where we are replacing with Duration and
> Instant, this is not a user specified value as part of the DSL but rather a
> passed-in parameter, plus with high punctuation frequency creating a new
> instance of Instant may be costly.
>
> 2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore with
> APIs of `long` as well as inheriting APIs of `Duration`.
>
>
> Guozhang
>
>
> On Mon, Sep 10, 2018 at 11:11 AM, Nikolay Izhikov 
> wrote:
>
> > Hello, Matthias.
> >
> > > (4) While I agree that we might want to deprecate it, I am not sure if
> > this should be part of this KIP?
> > > Seems to be unrelated?
> > > Should this have been part of KIP-319?
> > > If yes, we might still want to updated this other KIP? WDYT?
> >
> > OK, I removed this deprecation from this KIP.
> >
> > Please, tell me, is there anything else that should be improved to make
> > this KIP ready to be implemented.
> >
> > В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
> > > (1) Sounds good to me, to just use IllegalArgumentException for both --
> > > and thanks for pointing out that Duration can be negative and we need
> to
> > > check for this. For the KIP, it would be nice to add to all methods
> than
> > > (even if we don't do it in the code but only document in JavaDocs).
> > >
> > > (2) I would argue for a new single method interface. Not sure about the
> > > name though.
> > >
> > > (3) Even if `#fetch(K, K, long, long)` and `#fetchAll(long, long)` is
> > > _currently_ not used internally, I would still argue they are both dual
> > > use -- we might all a new DSL operator at any point that uses those
> > > methods. Thus to be "future prove" I would consider them dual use.
> > >
> > > > Since the ReadOnlyWindowStore is only used by IQ,
> > >
> > > This contradicts your other statement:
> > >
> > > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is
> > used
> > > > in KStreamWindowAggregate.
> > >
> > > Or do you suggest to move `fetch(K, long)` from `ReadOnlyWindowStore`
> to
> > > `WindowStore` ? This would not make sense IMHO, as `WindowStore extends
> > > ReadOnlyWindowStore` and thus, we would loose this method for IQ.
> > >
> > >
> > > (4) While I agree that we might want to deprecate it, I am not sure if
> > > this should be part of this KIP? Seems to be unrelated? Should this
> have
> > > been part of KIP-319? If yes, we might still want to updated this other
> > > KIP? WDYT?
> > >
> > >
> > > -Matthias
> > >
> > >
> > > On 9/7/18 12:09 PM, John Roesler wrote:
> > > > Hey all,
> > > >
> > > > (1): Duration can be negative, just like long. We need to enforce any
> > > > bounds that we currently enforce. We don't need the `throws`
> > declaration
> > > > for runtime exceptions, but the potential IllegalArgumentException
> > should
> > > > be documented in the javadoc for these methods. I still feel that
> > surfacing
> > > > the ArithmeticException directly would not be a great experience, so
> I
> > > > still advocate for wrapping it in an IllegalArgumentException that
> > explains
> > > > our upper bound for Duration is "max-long number of milliseconds"
> > > >
> > > > (2): I agree with your performance intuition. I don't think creating
> > one
> > > > object per call to punctuate is going to substantially affect the
> > > > performance.
> > > >
> > > > I think the deeper problem with Punctuator is that it's currently a
> SAM
> > > > interface. If we add a new method to it, we break the source code of
> > anyone
> > > > passing a function. We can add the new method with a default
> > > > implementation, as Nikolay suggested, but then you get into figuring
> > out
> > > > which one to default, and no one's happy. Alternatively, we can just
> > make a
> > > > brand new interface that is still a single method (but an Instant)
> and
> > add
> > > > the appropriate overloads and deprecate the old ones.
> > > >
> > > > (3): I disagree. I think only two methods are dual use, and we should
> > > > separate the internal from external usages. The internal usage should
> > be
> > > > added to WindowStore.
> > > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is
> > used
> > > > in KStreamWindowAggregate.
> > > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long,
> > long) is
> > > > used in KStreamKStreamJoin.
> > > > Both of these usages are as WindowStore, so adding these interfaces
> to
> > > > WindowStore would be transparent.
> > > >
> > > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, K, long,
> > long)
> > > > is only used for IQ
> > > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetchAll(long,
> > long) is
> > > > only used 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Guozhang Wang]

Hello Nikolay,

Thanks for picking this up! Just sharing my two cents:

1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is since
comparing with other places where we are replacing with Duration and
Instant, this is not a user specified value as part of the DSL but rather a
passed-in parameter, plus with high punctuation frequency creating a new
instance of Instant may be costly.

2) I'm fine with keeping KeyValueStore extending ReadOnlyKeyValueStore with
APIs of `long` as well as inheriting APIs of `Duration`.


Guozhang


On Mon, Sep 10, 2018 at 11:11 AM, Nikolay Izhikov 
wrote:

> Hello, Matthias.
>
> > (4) While I agree that we might want to deprecate it, I am not sure if
> this should be part of this KIP?
> > Seems to be unrelated?
> > Should this have been part of KIP-319?
> > If yes, we might still want to updated this other KIP? WDYT?
>
> OK, I removed this deprecation from this KIP.
>
> Please, tell me, is there anything else that should be improved to make
> this KIP ready to be implemented.
>
> В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
> > (1) Sounds good to me, to just use IllegalArgumentException for both --
> > and thanks for pointing out that Duration can be negative and we need to
> > check for this. For the KIP, it would be nice to add to all methods than
> > (even if we don't do it in the code but only document in JavaDocs).
> >
> > (2) I would argue for a new single method interface. Not sure about the
> > name though.
> >
> > (3) Even if `#fetch(K, K, long, long)` and `#fetchAll(long, long)` is
> > _currently_ not used internally, I would still argue they are both dual
> > use -- we might all a new DSL operator at any point that uses those
> > methods. Thus to be "future prove" I would consider them dual use.
> >
> > > Since the ReadOnlyWindowStore is only used by IQ,
> >
> > This contradicts your other statement:
> >
> > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is
> used
> > > in KStreamWindowAggregate.
> >
> > Or do you suggest to move `fetch(K, long)` from `ReadOnlyWindowStore` to
> > `WindowStore` ? This would not make sense IMHO, as `WindowStore extends
> > ReadOnlyWindowStore` and thus, we would loose this method for IQ.
> >
> >
> > (4) While I agree that we might want to deprecate it, I am not sure if
> > this should be part of this KIP? Seems to be unrelated? Should this have
> > been part of KIP-319? If yes, we might still want to updated this other
> > KIP? WDYT?
> >
> >
> > -Matthias
> >
> >
> > On 9/7/18 12:09 PM, John Roesler wrote:
> > > Hey all,
> > >
> > > (1): Duration can be negative, just like long. We need to enforce any
> > > bounds that we currently enforce. We don't need the `throws`
> declaration
> > > for runtime exceptions, but the potential IllegalArgumentException
> should
> > > be documented in the javadoc for these methods. I still feel that
> surfacing
> > > the ArithmeticException directly would not be a great experience, so I
> > > still advocate for wrapping it in an IllegalArgumentException that
> explains
> > > our upper bound for Duration is "max-long number of milliseconds"
> > >
> > > (2): I agree with your performance intuition. I don't think creating
> one
> > > object per call to punctuate is going to substantially affect the
> > > performance.
> > >
> > > I think the deeper problem with Punctuator is that it's currently a SAM
> > > interface. If we add a new method to it, we break the source code of
> anyone
> > > passing a function. We can add the new method with a default
> > > implementation, as Nikolay suggested, but then you get into figuring
> out
> > > which one to default, and no one's happy. Alternatively, we can just
> make a
> > > brand new interface that is still a single method (but an Instant) and
> add
> > > the appropriate overloads and deprecate the old ones.
> > >
> > > (3): I disagree. I think only two methods are dual use, and we should
> > > separate the internal from external usages. The internal usage should
> be
> > > added to WindowStore.
> > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is
> used
> > > in KStreamWindowAggregate.
> > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long,
> long) is
> > > used in KStreamKStreamJoin.
> > > Both of these usages are as WindowStore, so adding these interfaces to
> > > WindowStore would be transparent.
> > >
> > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, K, long,
> long)
> > > is only used for IQ
> > > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetchAll(long,
> long) is
> > > only used for IQ
> > >
> > > Since the ReadOnlyWindowStore is only used by IQ, we can safely say
> that
> > > *all* of its methods are external use and can be deprecated and
> replaced.
> > > The first two usages I noted are WindowStore usages, not
> > > ReadOnlyWindowStores, and WindowStore is only used *internally*, so
> it's
> > > free to offer `long` methods if needed for performance 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Matthias.

> (4) While I agree that we might want to deprecate it, I am not sure if this 
> should be part of this KIP? 
> Seems to be unrelated? 
> Should this have been part of KIP-319? 
> If yes, we might still want to updated this other KIP? WDYT?

OK, I removed this deprecation from this KIP.

Please, tell me, is there anything else that should be improved to make this 
KIP ready to be implemented.

В Пт, 07/09/2018 в 17:06 -0700, Matthias J. Sax пишет:
> (1) Sounds good to me, to just use IllegalArgumentException for both --
> and thanks for pointing out that Duration can be negative and we need to
> check for this. For the KIP, it would be nice to add to all methods than
> (even if we don't do it in the code but only document in JavaDocs).
> 
> (2) I would argue for a new single method interface. Not sure about the
> name though.
> 
> (3) Even if `#fetch(K, K, long, long)` and `#fetchAll(long, long)` is
> _currently_ not used internally, I would still argue they are both dual
> use -- we might all a new DSL operator at any point that uses those
> methods. Thus to be "future prove" I would consider them dual use.
> 
> > Since the ReadOnlyWindowStore is only used by IQ,
> 
> This contradicts your other statement:
> 
> > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is used
> > in KStreamWindowAggregate.
> 
> Or do you suggest to move `fetch(K, long)` from `ReadOnlyWindowStore` to
> `WindowStore` ? This would not make sense IMHO, as `WindowStore extends
> ReadOnlyWindowStore` and thus, we would loose this method for IQ.
> 
> 
> (4) While I agree that we might want to deprecate it, I am not sure if
> this should be part of this KIP? Seems to be unrelated? Should this have
> been part of KIP-319? If yes, we might still want to updated this other
> KIP? WDYT?
> 
> 
> -Matthias
> 
> 
> On 9/7/18 12:09 PM, John Roesler wrote:
> > Hey all,
> > 
> > (1): Duration can be negative, just like long. We need to enforce any
> > bounds that we currently enforce. We don't need the `throws` declaration
> > for runtime exceptions, but the potential IllegalArgumentException should
> > be documented in the javadoc for these methods. I still feel that surfacing
> > the ArithmeticException directly would not be a great experience, so I
> > still advocate for wrapping it in an IllegalArgumentException that explains
> > our upper bound for Duration is "max-long number of milliseconds"
> > 
> > (2): I agree with your performance intuition. I don't think creating one
> > object per call to punctuate is going to substantially affect the
> > performance.
> > 
> > I think the deeper problem with Punctuator is that it's currently a SAM
> > interface. If we add a new method to it, we break the source code of anyone
> > passing a function. We can add the new method with a default
> > implementation, as Nikolay suggested, but then you get into figuring out
> > which one to default, and no one's happy. Alternatively, we can just make a
> > brand new interface that is still a single method (but an Instant) and add
> > the appropriate overloads and deprecate the old ones.
> > 
> > (3): I disagree. I think only two methods are dual use, and we should
> > separate the internal from external usages. The internal usage should be
> > added to WindowStore.
> > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is used
> > in KStreamWindowAggregate.
> > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long, long) is
> > used in KStreamKStreamJoin.
> > Both of these usages are as WindowStore, so adding these interfaces to
> > WindowStore would be transparent.
> > 
> > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, K, long, long)
> > is only used for IQ
> > org.apache.kafka.streams.state.ReadOnlyWindowStore#fetchAll(long, long) is
> > only used for IQ
> > 
> > Since the ReadOnlyWindowStore is only used by IQ, we can safely say that
> > *all* of its methods are external use and can be deprecated and replaced.
> > The first two usages I noted are WindowStore usages, not
> > ReadOnlyWindowStores, and WindowStore is only used *internally*, so it's
> > free to offer `long` methods if needed for performance reasons.
> > 
> > Does this make sense? The same reasoning extends to the other stores.
> > 
> > (4) Yes, that was my suggestion. I'm not sure if anyone is actually using
> > this variant, so it seemed like a good time to just deprecate it and see.
> > 
> > Thoughts?
> > -John
> > 
> > 
> > On Fri, Sep 7, 2018 at 10:21 AM Nikolay Izhikov  wrote:
> > 
> > > Hello, Matthias.
> > > 
> > > Thanks, for feedback.
> > > 
> > > > (1) Some methods declare `throws IllegalArgumentException`, others>
> > > 
> > > don't.
> > > 
> > > `duration.toMillis()` can throw ArithmeticException.
> > > It can happen if overflow occurs during conversion.
> > > Please, see source of jdk method Duration#toMillis.
> > > Task author suggest to wrap it to IllegalArgumentException.
> > > I think we should add 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Matthias J. Sax]

(1) Sounds good to me, to just use IllegalArgumentException for both --
and thanks for pointing out that Duration can be negative and we need to
check for this. For the KIP, it would be nice to add to all methods than
(even if we don't do it in the code but only document in JavaDocs).

(2) I would argue for a new single method interface. Not sure about the
name though.

(3) Even if `#fetch(K, K, long, long)` and `#fetchAll(long, long)` is
_currently_ not used internally, I would still argue they are both dual
use -- we might all a new DSL operator at any point that uses those
methods. Thus to be "future prove" I would consider them dual use.

> Since the ReadOnlyWindowStore is only used by IQ,

This contradicts your other statement:

> org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is used
> in KStreamWindowAggregate.

Or do you suggest to move `fetch(K, long)` from `ReadOnlyWindowStore` to
`WindowStore` ? This would not make sense IMHO, as `WindowStore extends
ReadOnlyWindowStore` and thus, we would loose this method for IQ.


(4) While I agree that we might want to deprecate it, I am not sure if
this should be part of this KIP? Seems to be unrelated? Should this have
been part of KIP-319? If yes, we might still want to updated this other
KIP? WDYT?


-Matthias


On 9/7/18 12:09 PM, John Roesler wrote:
> Hey all,
> 
> (1): Duration can be negative, just like long. We need to enforce any
> bounds that we currently enforce. We don't need the `throws` declaration
> for runtime exceptions, but the potential IllegalArgumentException should
> be documented in the javadoc for these methods. I still feel that surfacing
> the ArithmeticException directly would not be a great experience, so I
> still advocate for wrapping it in an IllegalArgumentException that explains
> our upper bound for Duration is "max-long number of milliseconds"
> 
> (2): I agree with your performance intuition. I don't think creating one
> object per call to punctuate is going to substantially affect the
> performance.
> 
> I think the deeper problem with Punctuator is that it's currently a SAM
> interface. If we add a new method to it, we break the source code of anyone
> passing a function. We can add the new method with a default
> implementation, as Nikolay suggested, but then you get into figuring out
> which one to default, and no one's happy. Alternatively, we can just make a
> brand new interface that is still a single method (but an Instant) and add
> the appropriate overloads and deprecate the old ones.
> 
> (3): I disagree. I think only two methods are dual use, and we should
> separate the internal from external usages. The internal usage should be
> added to WindowStore.
> org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is used
> in KStreamWindowAggregate.
> org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long, long) is
> used in KStreamKStreamJoin.
> Both of these usages are as WindowStore, so adding these interfaces to
> WindowStore would be transparent.
> 
> org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, K, long, long)
> is only used for IQ
> org.apache.kafka.streams.state.ReadOnlyWindowStore#fetchAll(long, long) is
> only used for IQ
> 
> Since the ReadOnlyWindowStore is only used by IQ, we can safely say that
> *all* of its methods are external use and can be deprecated and replaced.
> The first two usages I noted are WindowStore usages, not
> ReadOnlyWindowStores, and WindowStore is only used *internally*, so it's
> free to offer `long` methods if needed for performance reasons.
> 
> Does this make sense? The same reasoning extends to the other stores.
> 
> (4) Yes, that was my suggestion. I'm not sure if anyone is actually using
> this variant, so it seemed like a good time to just deprecate it and see.
> 
> Thoughts?
> -John
> 
> 
> On Fri, Sep 7, 2018 at 10:21 AM Nikolay Izhikov  wrote:
> 
>> Hello, Matthias.
>>
>> Thanks, for feedback.
>>
>>> (1) Some methods declare `throws IllegalArgumentException`, others>
>> don't.
>>
>> `duration.toMillis()` can throw ArithmeticException.
>> It can happen if overflow occurs during conversion.
>> Please, see source of jdk method Duration#toMillis.
>> Task author suggest to wrap it to IllegalArgumentException.
>> I think we should add `throws IllegalArgumentException` for all method
>> with Duration parameter.
>> (I updated KIP with this throws)
>>
>> What do you think?
>>
>>> (3) ReadOnlyWindowStore: All three methods are dual use and I think we
>> should not deprecate them.
>>
>> This is my typo, already fixed.
>> I propose to add new methods to `ReadOnlyWindowStore`.
>> No methods will become deprecated.
>>
>>> (4) Stores: 3 methods are listed as deprecated but only 2 new methods
>> are added.
>>
>> My proposal based on John Roesler mail [1]:
>> "10. Stores: I think we can just deprecate without replacement the method
>> that takes `segmentInterval`."
>>
>> Is it wrong?
>>
>> [1] 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[John Roesler]

Hey all,

(1): Duration can be negative, just like long. We need to enforce any
bounds that we currently enforce. We don't need the `throws` declaration
for runtime exceptions, but the potential IllegalArgumentException should
be documented in the javadoc for these methods. I still feel that surfacing
the ArithmeticException directly would not be a great experience, so I
still advocate for wrapping it in an IllegalArgumentException that explains
our upper bound for Duration is "max-long number of milliseconds"

(2): I agree with your performance intuition. I don't think creating one
object per call to punctuate is going to substantially affect the
performance.

I think the deeper problem with Punctuator is that it's currently a SAM
interface. If we add a new method to it, we break the source code of anyone
passing a function. We can add the new method with a default
implementation, as Nikolay suggested, but then you get into figuring out
which one to default, and no one's happy. Alternatively, we can just make a
brand new interface that is still a single method (but an Instant) and add
the appropriate overloads and deprecate the old ones.

(3): I disagree. I think only two methods are dual use, and we should
separate the internal from external usages. The internal usage should be
added to WindowStore.
org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long) is used
in KStreamWindowAggregate.
org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, long, long) is
used in KStreamKStreamJoin.
Both of these usages are as WindowStore, so adding these interfaces to
WindowStore would be transparent.

org.apache.kafka.streams.state.ReadOnlyWindowStore#fetch(K, K, long, long)
is only used for IQ
org.apache.kafka.streams.state.ReadOnlyWindowStore#fetchAll(long, long) is
only used for IQ

Since the ReadOnlyWindowStore is only used by IQ, we can safely say that
*all* of its methods are external use and can be deprecated and replaced.
The first two usages I noted are WindowStore usages, not
ReadOnlyWindowStores, and WindowStore is only used *internally*, so it's
free to offer `long` methods if needed for performance reasons.

Does this make sense? The same reasoning extends to the other stores.

(4) Yes, that was my suggestion. I'm not sure if anyone is actually using
this variant, so it seemed like a good time to just deprecate it and see.

Thoughts?
-John


On Fri, Sep 7, 2018 at 10:21 AM Nikolay Izhikov  wrote:

> Hello, Matthias.
>
> Thanks, for feedback.
>
> > (1) Some methods declare `throws IllegalArgumentException`, others>
> don't.
>
> `duration.toMillis()` can throw ArithmeticException.
> It can happen if overflow occurs during conversion.
> Please, see source of jdk method Duration#toMillis.
> Task author suggest to wrap it to IllegalArgumentException.
> I think we should add `throws IllegalArgumentException` for all method
> with Duration parameter.
> (I updated KIP with this throws)
>
> What do you think?
>
> > (3) ReadOnlyWindowStore: All three methods are dual use and I think we
> should not deprecate them.
>
> This is my typo, already fixed.
> I propose to add new methods to `ReadOnlyWindowStore`.
> No methods will become deprecated.
>
> > (4) Stores: 3 methods are listed as deprecated but only 2 new methods
> are added.
>
> My proposal based on John Roesler mail [1]:
> "10. Stores: I think we can just deprecate without replacement the method
> that takes `segmentInterval`."
>
> Is it wrong?
>
> [1] https://www.mail-archive.com/dev@kafka.apache.org/msg91348.html
>
>
> В Чт, 06/09/2018 в 21:04 -0700, Matthias J. Sax пишет:
> > Thanks for updating the KIP!
> >
> > Couple of minor follow ups:
> >
> > (1) Some methods declare `throws IllegalArgumentException`, others
> > don't. It's runtime exception and thus it's not required to declare it
> > -- it just looks inconsistent in the KIP and maybe it's inconsistent in
> > the code, too. I am not sure if it is possible to provide a negative
> > Duration? If not, we would not need to check the provided value and can
> > remove the declaration.
> >
> > (2) About punctuations: I still think, it would be ok to change the
> > callback from `long` to `Instance` -- even if it is possible to register
> > a punctuation on a ms-basis, in practice many people used schedules in
> > the range of seconds or larger. Thus, I don't think there will be a
> > performance penalty. Of course, we can still revisit this later, too.
> > John and Bill, you did not comment on this. Would also be good to get
> > feedback from Guozhang about this.
> >
> > (3) ReadOnlyWindowStore: All three methods are dual use and I think we
> > should not deprecate them. However, we can add the new proposed methods
> > in parallel -- the names can be the same without conflict as the
> > parameter lists are different. (Or did you just forget to remove the
> > comment line?)
> >
> > (4) Stores: 3 methods are listed as deprecated but only 2 new methods
> > are added. Maybe this was discussed 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Matthias.

Thanks, for feedback.

> (1) Some methods declare `throws IllegalArgumentException`, others> don't.

`duration.toMillis()` can throw ArithmeticException.
It can happen if overflow occurs during conversion.
Please, see source of jdk method Duration#toMillis.
Task author suggest to wrap it to IllegalArgumentException.
I think we should add `throws IllegalArgumentException` for all method with 
Duration parameter.
(I updated KIP with this throws)

What do you think?

> (3) ReadOnlyWindowStore: All three methods are dual use and I think we should 
> not deprecate them.

This is my typo, already fixed.
I propose to add new methods to `ReadOnlyWindowStore`.
No methods will become deprecated.

> (4) Stores: 3 methods are listed as deprecated but only 2 new methods are 
> added.

My proposal based on John Roesler mail [1]:
"10. Stores: I think we can just deprecate without replacement the method that 
takes `segmentInterval`."

Is it wrong?

[1] https://www.mail-archive.com/dev@kafka.apache.org/msg91348.html


В Чт, 06/09/2018 в 21:04 -0700, Matthias J. Sax пишет:
> Thanks for updating the KIP!
> 
> Couple of minor follow ups:
> 
> (1) Some methods declare `throws IllegalArgumentException`, others
> don't. It's runtime exception and thus it's not required to declare it
> -- it just looks inconsistent in the KIP and maybe it's inconsistent in
> the code, too. I am not sure if it is possible to provide a negative
> Duration? If not, we would not need to check the provided value and can
> remove the declaration.
> 
> (2) About punctuations: I still think, it would be ok to change the
> callback from `long` to `Instance` -- even if it is possible to register
> a punctuation on a ms-basis, in practice many people used schedules in
> the range of seconds or larger. Thus, I don't think there will be a
> performance penalty. Of course, we can still revisit this later, too.
> John and Bill, you did not comment on this. Would also be good to get
> feedback from Guozhang about this.
> 
> (3) ReadOnlyWindowStore: All three methods are dual use and I think we
> should not deprecate them. However, we can add the new proposed methods
> in parallel -- the names can be the same without conflict as the
> parameter lists are different. (Or did you just forget to remove the
> comment line?)
> 
> (4) Stores: 3 methods are listed as deprecated but only 2 new methods
> are added. Maybe this was discussed already, but I can't recall why? Can
> you elaborate? Or should this deprecation be actually be part of KIP-328
> (\cc John)?
> 
> 
> Thanks,
> 
> -Matthias
> 
> 
> 
> ps: there are many KIPs in-flight in parallel, and it takes some time to
> get around. Please be patient :)
> 
> 
> 
> 
> On 9/5/18 12:25 AM, Nikolay Izhikov wrote:
> > Hello, Guys.
> > 
> > I've started a VOTE [1], but seems commiters have no chance to look at KIP 
> > for now.
> > 
> > Can you tell me, is it OK?
> > Should I wait for feedback? For how long?
> > 
> > Or something in KIP should be improved before voting?
> > 
> > [1] 
> > https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > 
> > В Пт, 24/08/2018 в 10:36 -0700, Matthias J. Sax пишет:
> > > It's tricky... :)
> > > 
> > > Some APIs have "dual use" as I mentioned in my first reply. I agree that
> > > it would be good to avoid abstract class and use interfaces if possible.
> > > As long as the change is source code compatible, it should be fine IMHO
> > > -- we need to document binary incompatibility of course.
> > > 
> > > I think it's best, if the KIPs gets update with a proposal on how to
> > > handle "dual use" parts. It's easier to discuss if it's written down IMHO.
> > > 
> > > For `ProcessorContext#schedule()`, you are right John: it's seems fine
> > > to use `Duration`, as it won't be called often (usually only within
> > > `Processor#init()`) -- I mixed it up with `Punctuator#punctuate(long)`.
> > > However, thinking about this twice, we might even want to update both
> > > methods. Punctuation callbacks don't happen every millisecond and thus
> > > the overhead to use `Instance` should not be a problem.
> > > 
> > > @Nikolay: it seems the KIP does not mention `Punctuator#punctuate(long)`
> > > -- should we add it?
> > > 
> > > 
> > > -Matthias
> > > 
> > > 
> > > On 8/24/18 10:11 AM, John Roesler wrote:
> > > > Quick afterthought: I guess that `Window` is exposed to the API via
> > > > `Windowed` keys. I think it would be fine to not deprecate the `long` 
> > > > start
> > > > and end, but add `Instant` variants for people preferring that 
> > > > interface.
> > > > 
> > > > On Fri, Aug 24, 2018 at 11:10 AM John Roesler  wrote:
> > > > 
> > > > > Hey Matthias,
> > > > > 
> > > > > Thanks for pointing that out. I agree that we only really need to 
> > > > > change
> > > > > methods that are API-facing, and we probably want to avoid using
> > > > > Duration/Instant for Streams-facing members.
> > > > > 
> > > > > 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Matthias J. Sax]

Thanks for updating the KIP!

Couple of minor follow ups:

(1) Some methods declare `throws IllegalArgumentException`, others
don't. It's runtime exception and thus it's not required to declare it
-- it just looks inconsistent in the KIP and maybe it's inconsistent in
the code, too. I am not sure if it is possible to provide a negative
Duration? If not, we would not need to check the provided value and can
remove the declaration.

(2) About punctuations: I still think, it would be ok to change the
callback from `long` to `Instance` -- even if it is possible to register
a punctuation on a ms-basis, in practice many people used schedules in
the range of seconds or larger. Thus, I don't think there will be a
performance penalty. Of course, we can still revisit this later, too.
John and Bill, you did not comment on this. Would also be good to get
feedback from Guozhang about this.

(3) ReadOnlyWindowStore: All three methods are dual use and I think we
should not deprecate them. However, we can add the new proposed methods
in parallel -- the names can be the same without conflict as the
parameter lists are different. (Or did you just forget to remove the
comment line?)

(4) Stores: 3 methods are listed as deprecated but only 2 new methods
are added. Maybe this was discussed already, but I can't recall why? Can
you elaborate? Or should this deprecation be actually be part of KIP-328
(\cc John)?


Thanks,

-Matthias



ps: there are many KIPs in-flight in parallel, and it takes some time to
get around. Please be patient :)




On 9/5/18 12:25 AM, Nikolay Izhikov wrote:
> Hello, Guys.
> 
> I've started a VOTE [1], but seems commiters have no chance to look at KIP 
> for now.
> 
> Can you tell me, is it OK?
> Should I wait for feedback? For how long?
> 
> Or something in KIP should be improved before voting?
> 
> [1] 
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> 
> В Пт, 24/08/2018 в 10:36 -0700, Matthias J. Sax пишет:
>> It's tricky... :)
>>
>> Some APIs have "dual use" as I mentioned in my first reply. I agree that
>> it would be good to avoid abstract class and use interfaces if possible.
>> As long as the change is source code compatible, it should be fine IMHO
>> -- we need to document binary incompatibility of course.
>>
>> I think it's best, if the KIPs gets update with a proposal on how to
>> handle "dual use" parts. It's easier to discuss if it's written down IMHO.
>>
>> For `ProcessorContext#schedule()`, you are right John: it's seems fine
>> to use `Duration`, as it won't be called often (usually only within
>> `Processor#init()`) -- I mixed it up with `Punctuator#punctuate(long)`.
>> However, thinking about this twice, we might even want to update both
>> methods. Punctuation callbacks don't happen every millisecond and thus
>> the overhead to use `Instance` should not be a problem.
>>
>> @Nikolay: it seems the KIP does not mention `Punctuator#punctuate(long)`
>> -- should we add it?
>>
>>
>> -Matthias
>>
>>
>> On 8/24/18 10:11 AM, John Roesler wrote:
>>> Quick afterthought: I guess that `Window` is exposed to the API via
>>> `Windowed` keys. I think it would be fine to not deprecate the `long` start
>>> and end, but add `Instant` variants for people preferring that interface.
>>>
>>> On Fri, Aug 24, 2018 at 11:10 AM John Roesler  wrote:
>>>
 Hey Matthias,

 Thanks for pointing that out. I agree that we only really need to change
 methods that are API-facing, and we probably want to avoid using
 Duration/Instant for Streams-facing members.

 Like I said in my last email, I think the whole Windows interface is
 Streams-facing, and the builders we provide are otherwise API-facing.
 Likewise, `Window` is Streams-facing, so start and end should not use
 Duration. In SessionWindows, inactivityGap is Streams-facing.

 I actually think that ProcessorContext#schedule() is API-facing, so it
 should use Duration. The rationale is that streams processing doesn't call
 this method, only implementer of Processor do. Does that seem right?

 Also, it seems like  ReadOnlyWindowStore#fetch() (2x) and #fetchAll() are
 API-facing (for IQ). When we call fetch() during processing, it's actually
 `WindowStore#fetch()`. Maybe we should move "WindowStoreIterator fetch(K
 key, long timeFrom, long timeTo)" to the WindowStore interface and make
 all the ReadOnlyWindowStore methods take Durations. And likewise with the
 SessionStore interfaces.

 What do you think?

 Thanks,
 -John




 On Fri, Aug 24, 2018 at 10:51 AM John Roesler  wrote:

> Hi Nikolay,
>
> First: I wanted to let you know that we have dropped the `grace(long)`
> method from the Windows interface, but we do still need to transition the
> same method on TimeWindows and JoinWindows (
> https://github.com/apache/kafka/pull/5536)
>

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Guys.

I've started a VOTE [1], but seems commiters have no chance to look at KIP for 
now.

Can you tell me, is it OK?
Should I wait for feedback? For how long?

Or something in KIP should be improved before voting?

[1] 
https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E

В Пт, 24/08/2018 в 10:36 -0700, Matthias J. Sax пишет:
> It's tricky... :)
> 
> Some APIs have "dual use" as I mentioned in my first reply. I agree that
> it would be good to avoid abstract class and use interfaces if possible.
> As long as the change is source code compatible, it should be fine IMHO
> -- we need to document binary incompatibility of course.
> 
> I think it's best, if the KIPs gets update with a proposal on how to
> handle "dual use" parts. It's easier to discuss if it's written down IMHO.
> 
> For `ProcessorContext#schedule()`, you are right John: it's seems fine
> to use `Duration`, as it won't be called often (usually only within
> `Processor#init()`) -- I mixed it up with `Punctuator#punctuate(long)`.
> However, thinking about this twice, we might even want to update both
> methods. Punctuation callbacks don't happen every millisecond and thus
> the overhead to use `Instance` should not be a problem.
> 
> @Nikolay: it seems the KIP does not mention `Punctuator#punctuate(long)`
> -- should we add it?
> 
> 
> -Matthias
> 
> 
> On 8/24/18 10:11 AM, John Roesler wrote:
> > Quick afterthought: I guess that `Window` is exposed to the API via
> > `Windowed` keys. I think it would be fine to not deprecate the `long` start
> > and end, but add `Instant` variants for people preferring that interface.
> > 
> > On Fri, Aug 24, 2018 at 11:10 AM John Roesler  wrote:
> > 
> > > Hey Matthias,
> > > 
> > > Thanks for pointing that out. I agree that we only really need to change
> > > methods that are API-facing, and we probably want to avoid using
> > > Duration/Instant for Streams-facing members.
> > > 
> > > Like I said in my last email, I think the whole Windows interface is
> > > Streams-facing, and the builders we provide are otherwise API-facing.
> > > Likewise, `Window` is Streams-facing, so start and end should not use
> > > Duration. In SessionWindows, inactivityGap is Streams-facing.
> > > 
> > > I actually think that ProcessorContext#schedule() is API-facing, so it
> > > should use Duration. The rationale is that streams processing doesn't call
> > > this method, only implementer of Processor do. Does that seem right?
> > > 
> > > Also, it seems like  ReadOnlyWindowStore#fetch() (2x) and #fetchAll() are
> > > API-facing (for IQ). When we call fetch() during processing, it's actually
> > > `WindowStore#fetch()`. Maybe we should move "WindowStoreIterator 
> > > fetch(K
> > > key, long timeFrom, long timeTo)" to the WindowStore interface and make
> > > all the ReadOnlyWindowStore methods take Durations. And likewise with the
> > > SessionStore interfaces.
> > > 
> > > What do you think?
> > > 
> > > Thanks,
> > > -John
> > > 
> > > 
> > > 
> > > 
> > > On Fri, Aug 24, 2018 at 10:51 AM John Roesler  wrote:
> > > 
> > > > Hi Nikolay,
> > > > 
> > > > First: I wanted to let you know that we have dropped the `grace(long)`
> > > > method from the Windows interface, but we do still need to transition 
> > > > the
> > > > same method on TimeWindows and JoinWindows (
> > > > https://github.com/apache/kafka/pull/5536)
> > > > 
> > > > I have also been thinking it would be nice to replace `Windows` with an
> > > > interface, but for different reasons. I think we can even do it without
> > > > breaking source compatibility (but it would break binary compatibility):
> > > > create a new interface `WindowSpec`, deprecate `Windows` and make it
> > > > implement `WindowSpec`, add a new method:
> > > > `KGroupedStream#windowedBy(WindowSpec)`, and deprecate the old one.
> > > > 
> > > > However, I don't think this would solve your problem, since the Windows
> > > > interface has two audiences: the DSL user and the implementer who 
> > > > wishes to
> > > > provide a new kind of windowing. I think we want to provide Duration to 
> > > > the
> > > > former, and long or Duration is fine for the latter. However, both of 
> > > > these
> > > > audiences are "external", so having an "internal" interface won't fit 
> > > > the
> > > > bill.
> > > > 
> > > > I think my last PR #5536 actually helps the situation quite a bit. Let's
> > > > forget about the deprecated members. Now, all the public members of 
> > > > Windows
> > > > are abstract methods, so Windows is effectively an interface now. Here's
> > > > how it looks:
> > > > 
> > > > public abstract class Windows {
> > > > public abstract Map windowsFor(final long timestamp);
> > > > public abstract long size();
> > > > public abstract long gracePeriodMs();
> > > > }
> > > > 
> > > > Notice that there is no part of this involved with the DSL. When you're
> > > > writing a topology, you don't call any of these methods. 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, 

I want to start VOTE for this KIP today.
Any objections?

В Пн, 27/08/2018 в 10:20 +0300, Nikolay Izhikov пишет:
> Hello, Matthias, John.
> 
> Thanks in advance.
> 
> > I wanted to let you know that we have dropped the `grace(long)` method from 
> > the Windows interface
> 
> `grace(long)` removed from the KIP.
> 
> > It seems like, if we want to use long millis internally, then we just need 
> > to leave Windows alone.
> 
> `Windows` removed from proposed API changes.
> 
> > In SessionWindows, inactivityGap is Streams-facing.
> 
> `inactivityGap` removed from proposed API changes.
> 
> > it seems the KIP does not mention `Punctuator#punctuate(long)` should we 
> > add it?
> 
> Actually, I think we shouldn't do it.
> 
> 1. If I understand correctly, user callback may be called every 1 millisecond 
> and many callbacks can be instantiated.
> Do we want to wrap every `long timestamp` into Instant in that case?
> 
> 2. If we introduce a new method `Punctuator.punctuate(Instant timestamp` 
> we should either break backward compatibility with new interface method or 
> provide default implementation:
> 
> public interface Punctuator {
> void punctuate(Instant timestmp);
> 
> default void punctuate(Instant timestamp) {
> punctuate(timestamp.toEpochMilli());
> }
> }
> 
> This doesn't seem right to me.
> What do you think?
> 
> > I think it's best, if the KIPs gets update with a proposal on how to handle 
> > "dual use" parts. 
> > It's easier to discuss if it's written down IMHO.
> 
> My proposal(copy of "Proposed Changes" section from KIP):
> 
> For the methods that used both: internally and as a part of public API the 
> proposal is:
> 
>   1. In this scope keep existing methods as is. 
>  Try to reduce the visibility of methods in next tickets.
>   2. Introduce finer methods with Instant and Duration.
> 
> В Пт, 24/08/2018 в 10:36 -0700, Matthias J. Sax пишет:
> > It's tricky... :)
> > 
> > Some APIs have "dual use" as I mentioned in my first reply. I agree that
> > it would be good to avoid abstract class and use interfaces if possible.
> > As long as the change is source code compatible, it should be fine IMHO
> > -- we need to document binary incompatibility of course.
> > 
> > I think it's best, if the KIPs gets update with a proposal on how to
> > handle "dual use" parts. It's easier to discuss if it's written down IMHO.
> > 
> > For `ProcessorContext#schedule()`, you are right John: it's seems fine
> > to use `Duration`, as it won't be called often (usually only within
> > `Processor#init()`) -- I mixed it up with `Punctuator#punctuate(long)`.
> > However, thinking about this twice, we might even want to update both
> > methods. Punctuation callbacks don't happen every millisecond and thus
> > the overhead to use `Instance` should not be a problem.
> > 
> > @Nikolay: it seems the KIP does not mention `Punctuator#punctuate(long)`
> > -- should we add it?
> > 
> > 
> > -Matthias
> > 
> > 
> > On 8/24/18 10:11 AM, John Roesler wrote:
> > > Quick afterthought: I guess that `Window` is exposed to the API via
> > > `Windowed` keys. I think it would be fine to not deprecate the `long` 
> > > start
> > > and end, but add `Instant` variants for people preferring that interface.
> > > 
> > > On Fri, Aug 24, 2018 at 11:10 AM John Roesler  wrote:
> > > 
> > > > Hey Matthias,
> > > > 
> > > > Thanks for pointing that out. I agree that we only really need to change
> > > > methods that are API-facing, and we probably want to avoid using
> > > > Duration/Instant for Streams-facing members.
> > > > 
> > > > Like I said in my last email, I think the whole Windows interface is
> > > > Streams-facing, and the builders we provide are otherwise API-facing.
> > > > Likewise, `Window` is Streams-facing, so start and end should not use
> > > > Duration. In SessionWindows, inactivityGap is Streams-facing.
> > > > 
> > > > I actually think that ProcessorContext#schedule() is API-facing, so it
> > > > should use Duration. The rationale is that streams processing doesn't 
> > > > call
> > > > this method, only implementer of Processor do. Does that seem right?
> > > > 
> > > > Also, it seems like  ReadOnlyWindowStore#fetch() (2x) and #fetchAll() 
> > > > are
> > > > API-facing (for IQ). When we call fetch() during processing, it's 
> > > > actually
> > > > `WindowStore#fetch()`. Maybe we should move "WindowStoreIterator 
> > > > fetch(K
> > > > key, long timeFrom, long timeTo)" to the WindowStore interface and make
> > > > all the ReadOnlyWindowStore methods take Durations. And likewise with 
> > > > the
> > > > SessionStore interfaces.
> > > > 
> > > > What do you think?
> > > > 
> > > > Thanks,
> > > > -John
> > > > 
> > > > 
> > > > 
> > > > 
> > > > On Fri, Aug 24, 2018 at 10:51 AM John Roesler  wrote:
> > > > 
> > > > > Hi Nikolay,
> > > > > 
> > > > > First: I wanted to let you know that we have dropped the `grace(long)`
> > > > > method from the Windows interface, but we do still 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Matthias, John.

Thanks in advance.

> I wanted to let you know that we have dropped the `grace(long)` method from 
> the Windows interface

`grace(long)` removed from the KIP.

> It seems like, if we want to use long millis internally, then we just need to 
> leave Windows alone.

`Windows` removed from proposed API changes.

> In SessionWindows, inactivityGap is Streams-facing.

`inactivityGap` removed from proposed API changes.

> it seems the KIP does not mention `Punctuator#punctuate(long)` should we add 
> it?

Actually, I think we shouldn't do it.

1. If I understand correctly, user callback may be called every 1 millisecond 
and many callbacks can be instantiated.
Do we want to wrap every `long timestamp` into Instant in that case?

2. If we introduce a new method `Punctuator.punctuate(Instant timestamp` 
we should either break backward compatibility with new interface method or 
provide default implementation:

public interface Punctuator {
void punctuate(Instant timestmp);

default void punctuate(Instant timestamp) {
punctuate(timestamp.toEpochMilli());
}
}

This doesn't seem right to me.
What do you think?

> I think it's best, if the KIPs gets update with a proposal on how to handle 
> "dual use" parts. 
> It's easier to discuss if it's written down IMHO.

My proposal(copy of "Proposed Changes" section from KIP):

For the methods that used both: internally and as a part of public API the 
proposal is:

1. In this scope keep existing methods as is. 
   Try to reduce the visibility of methods in next tickets.
2. Introduce finer methods with Instant and Duration.

В Пт, 24/08/2018 в 10:36 -0700, Matthias J. Sax пишет:
> It's tricky... :)
> 
> Some APIs have "dual use" as I mentioned in my first reply. I agree that
> it would be good to avoid abstract class and use interfaces if possible.
> As long as the change is source code compatible, it should be fine IMHO
> -- we need to document binary incompatibility of course.
> 
> I think it's best, if the KIPs gets update with a proposal on how to
> handle "dual use" parts. It's easier to discuss if it's written down IMHO.
> 
> For `ProcessorContext#schedule()`, you are right John: it's seems fine
> to use `Duration`, as it won't be called often (usually only within
> `Processor#init()`) -- I mixed it up with `Punctuator#punctuate(long)`.
> However, thinking about this twice, we might even want to update both
> methods. Punctuation callbacks don't happen every millisecond and thus
> the overhead to use `Instance` should not be a problem.
> 
> @Nikolay: it seems the KIP does not mention `Punctuator#punctuate(long)`
> -- should we add it?
> 
> 
> -Matthias
> 
> 
> On 8/24/18 10:11 AM, John Roesler wrote:
> > Quick afterthought: I guess that `Window` is exposed to the API via
> > `Windowed` keys. I think it would be fine to not deprecate the `long` start
> > and end, but add `Instant` variants for people preferring that interface.
> > 
> > On Fri, Aug 24, 2018 at 11:10 AM John Roesler  wrote:
> > 
> > > Hey Matthias,
> > > 
> > > Thanks for pointing that out. I agree that we only really need to change
> > > methods that are API-facing, and we probably want to avoid using
> > > Duration/Instant for Streams-facing members.
> > > 
> > > Like I said in my last email, I think the whole Windows interface is
> > > Streams-facing, and the builders we provide are otherwise API-facing.
> > > Likewise, `Window` is Streams-facing, so start and end should not use
> > > Duration. In SessionWindows, inactivityGap is Streams-facing.
> > > 
> > > I actually think that ProcessorContext#schedule() is API-facing, so it
> > > should use Duration. The rationale is that streams processing doesn't call
> > > this method, only implementer of Processor do. Does that seem right?
> > > 
> > > Also, it seems like  ReadOnlyWindowStore#fetch() (2x) and #fetchAll() are
> > > API-facing (for IQ). When we call fetch() during processing, it's actually
> > > `WindowStore#fetch()`. Maybe we should move "WindowStoreIterator 
> > > fetch(K
> > > key, long timeFrom, long timeTo)" to the WindowStore interface and make
> > > all the ReadOnlyWindowStore methods take Durations. And likewise with the
> > > SessionStore interfaces.
> > > 
> > > What do you think?
> > > 
> > > Thanks,
> > > -John
> > > 
> > > 
> > > 
> > > 
> > > On Fri, Aug 24, 2018 at 10:51 AM John Roesler  wrote:
> > > 
> > > > Hi Nikolay,
> > > > 
> > > > First: I wanted to let you know that we have dropped the `grace(long)`
> > > > method from the Windows interface, but we do still need to transition 
> > > > the
> > > > same method on TimeWindows and JoinWindows (
> > > > https://github.com/apache/kafka/pull/5536)
> > > > 
> > > > I have also been thinking it would be nice to replace `Windows` with an
> > > > interface, but for different reasons. I think we can even do it without
> > > > breaking source compatibility (but it would break binary compatibility):
> > > > create a new 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Matthias J. Sax]

It's tricky... :)

Some APIs have "dual use" as I mentioned in my first reply. I agree that
it would be good to avoid abstract class and use interfaces if possible.
As long as the change is source code compatible, it should be fine IMHO
-- we need to document binary incompatibility of course.

I think it's best, if the KIPs gets update with a proposal on how to
handle "dual use" parts. It's easier to discuss if it's written down IMHO.

For `ProcessorContext#schedule()`, you are right John: it's seems fine
to use `Duration`, as it won't be called often (usually only within
`Processor#init()`) -- I mixed it up with `Punctuator#punctuate(long)`.
However, thinking about this twice, we might even want to update both
methods. Punctuation callbacks don't happen every millisecond and thus
the overhead to use `Instance` should not be a problem.

@Nikolay: it seems the KIP does not mention `Punctuator#punctuate(long)`
-- should we add it?


-Matthias


On 8/24/18 10:11 AM, John Roesler wrote:
> Quick afterthought: I guess that `Window` is exposed to the API via
> `Windowed` keys. I think it would be fine to not deprecate the `long` start
> and end, but add `Instant` variants for people preferring that interface.
> 
> On Fri, Aug 24, 2018 at 11:10 AM John Roesler  wrote:
> 
>> Hey Matthias,
>>
>> Thanks for pointing that out. I agree that we only really need to change
>> methods that are API-facing, and we probably want to avoid using
>> Duration/Instant for Streams-facing members.
>>
>> Like I said in my last email, I think the whole Windows interface is
>> Streams-facing, and the builders we provide are otherwise API-facing.
>> Likewise, `Window` is Streams-facing, so start and end should not use
>> Duration. In SessionWindows, inactivityGap is Streams-facing.
>>
>> I actually think that ProcessorContext#schedule() is API-facing, so it
>> should use Duration. The rationale is that streams processing doesn't call
>> this method, only implementer of Processor do. Does that seem right?
>>
>> Also, it seems like  ReadOnlyWindowStore#fetch() (2x) and #fetchAll() are
>> API-facing (for IQ). When we call fetch() during processing, it's actually
>> `WindowStore#fetch()`. Maybe we should move "WindowStoreIterator fetch(K
>> key, long timeFrom, long timeTo)" to the WindowStore interface and make
>> all the ReadOnlyWindowStore methods take Durations. And likewise with the
>> SessionStore interfaces.
>>
>> What do you think?
>>
>> Thanks,
>> -John
>>
>>
>>
>>
>> On Fri, Aug 24, 2018 at 10:51 AM John Roesler  wrote:
>>
>>> Hi Nikolay,
>>>
>>> First: I wanted to let you know that we have dropped the `grace(long)`
>>> method from the Windows interface, but we do still need to transition the
>>> same method on TimeWindows and JoinWindows (
>>> https://github.com/apache/kafka/pull/5536)
>>>
>>> I have also been thinking it would be nice to replace `Windows` with an
>>> interface, but for different reasons. I think we can even do it without
>>> breaking source compatibility (but it would break binary compatibility):
>>> create a new interface `WindowSpec`, deprecate `Windows` and make it
>>> implement `WindowSpec`, add a new method:
>>> `KGroupedStream#windowedBy(WindowSpec)`, and deprecate the old one.
>>>
>>> However, I don't think this would solve your problem, since the Windows
>>> interface has two audiences: the DSL user and the implementer who wishes to
>>> provide a new kind of windowing. I think we want to provide Duration to the
>>> former, and long or Duration is fine for the latter. However, both of these
>>> audiences are "external", so having an "internal" interface won't fit the
>>> bill.
>>>
>>> I think my last PR #5536 actually helps the situation quite a bit. Let's
>>> forget about the deprecated members. Now, all the public members of Windows
>>> are abstract methods, so Windows is effectively an interface now. Here's
>>> how it looks:
>>>
>>> public abstract class Windows {
>>> public abstract Map windowsFor(final long timestamp);
>>> public abstract long size();
>>> public abstract long gracePeriodMs();
>>> }
>>>
>>> Notice that there is no part of this involved with the DSL. When you're
>>> writing a topology, you don't call any of these methods. It's strictly an
>>> interface that tells a Windows implementation what Streams expects from it.
>>> A very simple implementation could have no builder methods at all and just
>>> return fixed answers to these method calls (this is basically what
>>> UnlimitedWindows does). It seems like, if we want to use long millis
>>> internally, then we just need to leave Windows alone.
>>>
>>> What we do want to change is the builder methods in TimeWindows,
>>> JoinWindows, and UnlimitedWindows. For example, `TimeWindows#of(long)`
>>> would become `TimeWindows#of(Duration)`, etc. These are the DSL methods.
>>>
>>> Does that make sense?
>>> -John
>>>
>>>
>>>
>>> On Thu, Aug 23, 2018 at 8:59 AM Nikolay Izhikov 
>>> wrote:
>>>
 Hello, Mathias.

 Thanks for your 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[John Roesler]

Quick afterthought: I guess that `Window` is exposed to the API via
`Windowed` keys. I think it would be fine to not deprecate the `long` start
and end, but add `Instant` variants for people preferring that interface.

On Fri, Aug 24, 2018 at 11:10 AM John Roesler  wrote:

> Hey Matthias,
>
> Thanks for pointing that out. I agree that we only really need to change
> methods that are API-facing, and we probably want to avoid using
> Duration/Instant for Streams-facing members.
>
> Like I said in my last email, I think the whole Windows interface is
> Streams-facing, and the builders we provide are otherwise API-facing.
> Likewise, `Window` is Streams-facing, so start and end should not use
> Duration. In SessionWindows, inactivityGap is Streams-facing.
>
> I actually think that ProcessorContext#schedule() is API-facing, so it
> should use Duration. The rationale is that streams processing doesn't call
> this method, only implementer of Processor do. Does that seem right?
>
> Also, it seems like  ReadOnlyWindowStore#fetch() (2x) and #fetchAll() are
> API-facing (for IQ). When we call fetch() during processing, it's actually
> `WindowStore#fetch()`. Maybe we should move "WindowStoreIterator fetch(K
> key, long timeFrom, long timeTo)" to the WindowStore interface and make
> all the ReadOnlyWindowStore methods take Durations. And likewise with the
> SessionStore interfaces.
>
> What do you think?
>
> Thanks,
> -John
>
>
>
>
> On Fri, Aug 24, 2018 at 10:51 AM John Roesler  wrote:
>
>> Hi Nikolay,
>>
>> First: I wanted to let you know that we have dropped the `grace(long)`
>> method from the Windows interface, but we do still need to transition the
>> same method on TimeWindows and JoinWindows (
>> https://github.com/apache/kafka/pull/5536)
>>
>> I have also been thinking it would be nice to replace `Windows` with an
>> interface, but for different reasons. I think we can even do it without
>> breaking source compatibility (but it would break binary compatibility):
>> create a new interface `WindowSpec`, deprecate `Windows` and make it
>> implement `WindowSpec`, add a new method:
>> `KGroupedStream#windowedBy(WindowSpec)`, and deprecate the old one.
>>
>> However, I don't think this would solve your problem, since the Windows
>> interface has two audiences: the DSL user and the implementer who wishes to
>> provide a new kind of windowing. I think we want to provide Duration to the
>> former, and long or Duration is fine for the latter. However, both of these
>> audiences are "external", so having an "internal" interface won't fit the
>> bill.
>>
>> I think my last PR #5536 actually helps the situation quite a bit. Let's
>> forget about the deprecated members. Now, all the public members of Windows
>> are abstract methods, so Windows is effectively an interface now. Here's
>> how it looks:
>>
>> public abstract class Windows {
>> public abstract Map windowsFor(final long timestamp);
>> public abstract long size();
>> public abstract long gracePeriodMs();
>> }
>>
>> Notice that there is no part of this involved with the DSL. When you're
>> writing a topology, you don't call any of these methods. It's strictly an
>> interface that tells a Windows implementation what Streams expects from it.
>> A very simple implementation could have no builder methods at all and just
>> return fixed answers to these method calls (this is basically what
>> UnlimitedWindows does). It seems like, if we want to use long millis
>> internally, then we just need to leave Windows alone.
>>
>> What we do want to change is the builder methods in TimeWindows,
>> JoinWindows, and UnlimitedWindows. For example, `TimeWindows#of(long)`
>> would become `TimeWindows#of(Duration)`, etc. These are the DSL methods.
>>
>> Does that make sense?
>> -John
>>
>>
>>
>> On Thu, Aug 23, 2018 at 8:59 AM Nikolay Izhikov 
>> wrote:
>>
>>> Hello, Mathias.
>>>
>>> Thanks for your feedback.
>>>
>>> > Thus, it might make sense to keep old and just add new ones?
>>>
>>> As far as I understand, we will keep old methods anyway to prevent
>>> public API backward compatibility.
>>> I agree with you, methods that used internally shouldn't be deprecated.
>>>
>>> > End users can use the "nicer" new ones, while we can still use the
>>> existing ones internally?
>>> > Not sure if it would be possible to keep the old ones without exposing
>>> them as public API?
>>>
>>> I think, when we decide to remove methods with `long` from public API,
>>> we can do the following:
>>>
>>> 1. Create an interface like `WindowsInternal`.
>>> 2. Change Windows to an interface.
>>> 3. Create package-private implementation `WindowsImpl`.
>>>
>>> ```
>>> package org.apache.kafka.streams.kstream.internals;
>>> public interface WindowsInternal {
>>> public long start();
>>> public long end();
>>> //etc...
>>> }
>>>
>>> package org.apache.kafka.streams.kstream;
>>> public interface Windows {
>>> 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[John Roesler]

Hey Matthias,

Thanks for pointing that out. I agree that we only really need to change
methods that are API-facing, and we probably want to avoid using
Duration/Instant for Streams-facing members.

Like I said in my last email, I think the whole Windows interface is
Streams-facing, and the builders we provide are otherwise API-facing.
Likewise, `Window` is Streams-facing, so start and end should not use
Duration. In SessionWindows, inactivityGap is Streams-facing.

I actually think that ProcessorContext#schedule() is API-facing, so it
should use Duration. The rationale is that streams processing doesn't call
this method, only implementer of Processor do. Does that seem right?

Also, it seems like  ReadOnlyWindowStore#fetch() (2x) and #fetchAll() are
API-facing (for IQ). When we call fetch() during processing, it's actually
`WindowStore#fetch()`. Maybe we should move "WindowStoreIterator
fetch(K key,
long timeFrom, long timeTo)" to the WindowStore interface and make all the
ReadOnlyWindowStore methods take Durations. And likewise with the
SessionStore interfaces.

What do you think?

Thanks,
-John




On Fri, Aug 24, 2018 at 10:51 AM John Roesler  wrote:

> Hi Nikolay,
>
> First: I wanted to let you know that we have dropped the `grace(long)`
> method from the Windows interface, but we do still need to transition the
> same method on TimeWindows and JoinWindows (
> https://github.com/apache/kafka/pull/5536)
>
> I have also been thinking it would be nice to replace `Windows` with an
> interface, but for different reasons. I think we can even do it without
> breaking source compatibility (but it would break binary compatibility):
> create a new interface `WindowSpec`, deprecate `Windows` and make it
> implement `WindowSpec`, add a new method:
> `KGroupedStream#windowedBy(WindowSpec)`, and deprecate the old one.
>
> However, I don't think this would solve your problem, since the Windows
> interface has two audiences: the DSL user and the implementer who wishes to
> provide a new kind of windowing. I think we want to provide Duration to the
> former, and long or Duration is fine for the latter. However, both of these
> audiences are "external", so having an "internal" interface won't fit the
> bill.
>
> I think my last PR #5536 actually helps the situation quite a bit. Let's
> forget about the deprecated members. Now, all the public members of Windows
> are abstract methods, so Windows is effectively an interface now. Here's
> how it looks:
>
> public abstract class Windows {
> public abstract Map windowsFor(final long timestamp);
> public abstract long size();
> public abstract long gracePeriodMs();
> }
>
> Notice that there is no part of this involved with the DSL. When you're
> writing a topology, you don't call any of these methods. It's strictly an
> interface that tells a Windows implementation what Streams expects from it.
> A very simple implementation could have no builder methods at all and just
> return fixed answers to these method calls (this is basically what
> UnlimitedWindows does). It seems like, if we want to use long millis
> internally, then we just need to leave Windows alone.
>
> What we do want to change is the builder methods in TimeWindows,
> JoinWindows, and UnlimitedWindows. For example, `TimeWindows#of(long)`
> would become `TimeWindows#of(Duration)`, etc. These are the DSL methods.
>
> Does that make sense?
> -John
>
>
>
> On Thu, Aug 23, 2018 at 8:59 AM Nikolay Izhikov 
> wrote:
>
>> Hello, Mathias.
>>
>> Thanks for your feedback.
>>
>> > Thus, it might make sense to keep old and just add new ones?
>>
>> As far as I understand, we will keep old methods anyway to prevent public
>> API backward compatibility.
>> I agree with you, methods that used internally shouldn't be deprecated.
>>
>> > End users can use the "nicer" new ones, while we can still use the
>> existing ones internally?
>> > Not sure if it would be possible to keep the old ones without exposing
>> them as public API?
>>
>> I think, when we decide to remove methods with `long` from public API, we
>> can do the following:
>>
>> 1. Create an interface like `WindowsInternal`.
>> 2. Change Windows to an interface.
>> 3. Create package-private implementation `WindowsImpl`.
>>
>> ```
>> package org.apache.kafka.streams.kstream.internals;
>> public interface WindowsInternal {
>> public long start();
>> public long end();
>> //etc...
>> }
>>
>> package org.apache.kafka.streams.kstream;
>> public interface Windows {
>> public Instant start();
>> public Instant end();
>> //...
>> }
>>
>> class WindowsImpl implements Windows,
>> WindowsInternal {
>>
>> }
>> ```
>>
>> So, in public API we will expose only `Windows` interface and internally
>> we can use `WindowsInternal`
>> But, of course, this will be huge changes in public API.
>>
>> > Let me know what you think about 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[John Roesler]

Hi Nikolay,

First: I wanted to let you know that we have dropped the `grace(long)`
method from the Windows interface, but we do still need to transition the
same method on TimeWindows and JoinWindows (
https://github.com/apache/kafka/pull/5536)

I have also been thinking it would be nice to replace `Windows` with an
interface, but for different reasons. I think we can even do it without
breaking source compatibility (but it would break binary compatibility):
create a new interface `WindowSpec`, deprecate `Windows` and make it
implement `WindowSpec`, add a new method:
`KGroupedStream#windowedBy(WindowSpec)`, and deprecate the old one.

However, I don't think this would solve your problem, since the Windows
interface has two audiences: the DSL user and the implementer who wishes to
provide a new kind of windowing. I think we want to provide Duration to the
former, and long or Duration is fine for the latter. However, both of these
audiences are "external", so having an "internal" interface won't fit the
bill.

I think my last PR #5536 actually helps the situation quite a bit. Let's
forget about the deprecated members. Now, all the public members of Windows
are abstract methods, so Windows is effectively an interface now. Here's
how it looks:

public abstract class Windows {
public abstract Map windowsFor(final long timestamp);
public abstract long size();
public abstract long gracePeriodMs();
}

Notice that there is no part of this involved with the DSL. When you're
writing a topology, you don't call any of these methods. It's strictly an
interface that tells a Windows implementation what Streams expects from it.
A very simple implementation could have no builder methods at all and just
return fixed answers to these method calls (this is basically what
UnlimitedWindows does). It seems like, if we want to use long millis
internally, then we just need to leave Windows alone.

What we do want to change is the builder methods in TimeWindows,
JoinWindows, and UnlimitedWindows. For example, `TimeWindows#of(long)`
would become `TimeWindows#of(Duration)`, etc. These are the DSL methods.

Does that make sense?
-John



On Thu, Aug 23, 2018 at 8:59 AM Nikolay Izhikov  wrote:

> Hello, Mathias.
>
> Thanks for your feedback.
>
> > Thus, it might make sense to keep old and just add new ones?
>
> As far as I understand, we will keep old methods anyway to prevent public
> API backward compatibility.
> I agree with you, methods that used internally shouldn't be deprecated.
>
> > End users can use the "nicer" new ones, while we can still use the
> existing ones internally?
> > Not sure if it would be possible to keep the old ones without exposing
> them as public API?
>
> I think, when we decide to remove methods with `long` from public API, we
> can do the following:
>
> 1. Create an interface like `WindowsInternal`.
> 2. Change Windows to an interface.
> 3. Create package-private implementation `WindowsImpl`.
>
> ```
> package org.apache.kafka.streams.kstream.internals;
> public interface WindowsInternal {
> public long start();
> public long end();
> //etc...
> }
>
> package org.apache.kafka.streams.kstream;
> public interface Windows {
> public Instant start();
> public Instant end();
> //...
> }
>
> class WindowsImpl implements Windows,
> WindowsInternal {
>
> }
> ```
>
> So, in public API we will expose only `Windows` interface and internally
> we can use `WindowsInternal`
> But, of course, this will be huge changes in public API.
>
> > Let me know what you think about this.
>
> I think in this KIP we shouldn't deprecate methods, that are used
> internally.
> I changed it, now my proposal is just add new methods.
>
> Please, let me know if anything more need to be done.
>
> В Ср, 22/08/2018 в 17:29 -0700, Matthias J. Sax пишет:
> > Thanks a lot for the KIP.
> >
> > From my understanding, the idea of the KIP is to improve the public API
> > at DSL level. However, not all public methods listed are part of DSL
> > level API, but part of runtime API. Those methods are called during
> > processing and are on the hot code path. I am not sure, if we want to
> > update those methods. We should carefully think about this, and consider
> > to keep Long/long type to keep runtime overhead small. Note, that the
> > methods I mention are not required to specify a program using the DSL
> > and thus is questionable if the DSL API would be improved if we change
> > the methods.
> >
> > It's unfortunate, that some part of the public API stretch the DSL
> > builder part as well as the runtime part...
> >
> > This affects the following methods (please double check if I missed any):
> >
> >  - Windows#windowsFor()
> >  - Window#start()
> >  - Window#end()
> >  - JoinWindows#windowFor()
> >  - SessionWindows#inactivitiyGap()
> >  - TimeWindows#windowFor()
> >  - 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Mathias.

Thanks for your feedback.

> Thus, it might make sense to keep old and just add new ones? 

As far as I understand, we will keep old methods anyway to prevent public API 
backward compatibility.
I agree with you, methods that used internally shouldn't be deprecated.

> End users can use the "nicer" new ones, while we can still use the existing 
> ones internally? 
> Not sure if it would be possible to keep the old ones without exposing them 
> as public API?

I think, when we decide to remove methods with `long` from public API, we can 
do the following:

1. Create an interface like `WindowsInternal`.
2. Change Windows to an interface.
3. Create package-private implementation `WindowsImpl`.

```
package org.apache.kafka.streams.kstream.internals;
public interface WindowsInternal {
public long start();
public long end();
//etc...
}

package org.apache.kafka.streams.kstream;
public interface Windows {
public Instant start();
public Instant end();
//...
}

class WindowsImpl implements Windows, 
WindowsInternal {

}
```

So, in public API we will expose only `Windows` interface and internally we can 
use `WindowsInternal`
But, of course, this will be huge changes in public API.

> Let me know what you think about this.

I think in this KIP we shouldn't deprecate methods, that are used internally.
I changed it, now my proposal is just add new methods.

Please, let me know if anything more need to be done.

В Ср, 22/08/2018 в 17:29 -0700, Matthias J. Sax пишет:
> Thanks a lot for the KIP.
> 
> From my understanding, the idea of the KIP is to improve the public API
> at DSL level. However, not all public methods listed are part of DSL
> level API, but part of runtime API. Those methods are called during
> processing and are on the hot code path. I am not sure, if we want to
> update those methods. We should carefully think about this, and consider
> to keep Long/long type to keep runtime overhead small. Note, that the
> methods I mention are not required to specify a program using the DSL
> and thus is questionable if the DSL API would be improved if we change
> the methods.
> 
> It's unfortunate, that some part of the public API stretch the DSL
> builder part as well as the runtime part...
> 
> This affects the following methods (please double check if I missed any):
> 
>  - Windows#windowsFor()
>  - Window#start()
>  - Window#end()
>  - JoinWindows#windowFor()
>  - SessionWindows#inactivitiyGap()
>  - TimeWindows#windowFor()
>  - UnlimitedWindows#windowFor()
>  - ProcessorContext#schedule()
>  - ReadOnlyWindowStore#fetch() (2x) and #fetchAll()
>  - SessionStore#findSessions() (2x)
> 
> maybe
>  - TimeWindowedDeserializer#getWindowSize() (it's unused atm, but I
> could imagine that it might be use on the hot code path in the furture)
> 
> So methods have "dual" use and might be called externally and internally:
> 
>  - Window#start()
>  - Window#end()
>  - ReadOnlyWindowStore#fetch() (2x) and #fetchAll()
>  - SessionStore#findSessions() (2x)
> 
> Thus, it might make sense to keep old and just add new ones? End users
> can use the "nicer" new ones, while we can still use the existing ones
> internally? Not sure if it would be possible to keep the old ones
> without exposing them as public API?
> 
> Let me know what you think about this.
> 
> 
> -Matthias
> 
> 
> 
> On 8/21/18 11:41 PM, Nikolay Izhikov wrote:
> > Dear, commiters.
> > 
> > Please, pay attention to this KIP and share your opinion.
> > 
> > В Вт, 21/08/2018 в 11:14 -0500, John Roesler пишет:
> > > I'll solicit more reviews. Let's get at least one committer to chime in
> > > before we start a vote (since we need their approval anyway).
> > > -John
> > > 
> > > On Mon, Aug 20, 2018 at 12:39 PM Nikolay Izhikov 
> > > wrote:
> > > 
> > > > Hello, Ted.
> > > > 
> > > > Thanks for the comment.
> > > > 
> > > > I've edit KIP and change proposal to `windowSize`.
> > > > 
> > > > Guys, any other comments?
> > > > 
> > > > 
> > > > В Вс, 19/08/2018 в 14:57 -0700, Ted Yu пишет:
> > > > > bq. // or just Duration windowSize();
> > > > > 
> > > > > +1 to the above choice.
> > > > > The duration is obvious from the return type. For getter methods, we
> > > > 
> > > > don't
> > > > > use get as prefix (as least for new code).
> > > > > 
> > > > > Cheers
> > > > > 
> > > > > On Sun, Aug 19, 2018 at 8:03 AM Nikolay Izhikov 
> > > > 
> > > > wrote:
> > > > > 
> > > > > > Hello, John.
> > > > > > 
> > > > > > Thank you very much for your feedback!
> > > > > > I've addressed all your comments.
> > > > > > Please, see my answers and let my know is anything in KIP [1] needs 
> > > > > > to
> > > > 
> > > > be
> > > > > > improved.
> > > > > > 
> > > > > > > The correct choice is actually "Instant", not> "LocalDateTime"
> > > > > > 
> > > > > > I've changed the methods proposed in KIP [1] to use 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Matthias J. Sax]

Thanks a lot for the KIP.

From my understanding, the idea of the KIP is to improve the public API
at DSL level. However, not all public methods listed are part of DSL
level API, but part of runtime API. Those methods are called during
processing and are on the hot code path. I am not sure, if we want to
update those methods. We should carefully think about this, and consider
to keep Long/long type to keep runtime overhead small. Note, that the
methods I mention are not required to specify a program using the DSL
and thus is questionable if the DSL API would be improved if we change
the methods.

It's unfortunate, that some part of the public API stretch the DSL
builder part as well as the runtime part...

This affects the following methods (please double check if I missed any):

 - Windows#windowsFor()
 - Window#start()
 - Window#end()
 - JoinWindows#windowFor()
 - SessionWindows#inactivitiyGap()
 - TimeWindows#windowFor()
 - UnlimitedWindows#windowFor()
 - ProcessorContext#schedule()
 - ReadOnlyWindowStore#fetch() (2x) and #fetchAll()
 - SessionStore#findSessions() (2x)

maybe
 - TimeWindowedDeserializer#getWindowSize() (it's unused atm, but I
could imagine that it might be use on the hot code path in the furture)

So methods have "dual" use and might be called externally and internally:

 - Window#start()
 - Window#end()
 - ReadOnlyWindowStore#fetch() (2x) and #fetchAll()
 - SessionStore#findSessions() (2x)

Thus, it might make sense to keep old and just add new ones? End users
can use the "nicer" new ones, while we can still use the existing ones
internally? Not sure if it would be possible to keep the old ones
without exposing them as public API?

Let me know what you think about this.


-Matthias



On 8/21/18 11:41 PM, Nikolay Izhikov wrote:
> Dear, commiters.
> 
> Please, pay attention to this KIP and share your opinion.
> 
> В Вт, 21/08/2018 в 11:14 -0500, John Roesler пишет:
>> I'll solicit more reviews. Let's get at least one committer to chime in
>> before we start a vote (since we need their approval anyway).
>> -John
>>
>> On Mon, Aug 20, 2018 at 12:39 PM Nikolay Izhikov 
>> wrote:
>>
>>> Hello, Ted.
>>>
>>> Thanks for the comment.
>>>
>>> I've edit KIP and change proposal to `windowSize`.
>>>
>>> Guys, any other comments?
>>>
>>>
>>> В Вс, 19/08/2018 в 14:57 -0700, Ted Yu пишет:
 bq. // or just Duration windowSize();

 +1 to the above choice.
 The duration is obvious from the return type. For getter methods, we
>>>
>>> don't
 use get as prefix (as least for new code).

 Cheers

 On Sun, Aug 19, 2018 at 8:03 AM Nikolay Izhikov 
>>>
>>> wrote:

> Hello, John.
>
> Thank you very much for your feedback!
> I've addressed all your comments.
> Please, see my answers and let my know is anything in KIP [1] needs to
>>>
>>> be
> improved.
>
>> The correct choice is actually "Instant", not> "LocalDateTime"
>
> I've changed the methods proposed in KIP [1] to use Instant.
>
>> I noticed some recent APIs are> missing (see KIP-328)
>> those APIs were just added and have never been released... you can
>>>
>>> just
>
> replace them.
>
> I've added new methods to KIP [1].
> Not released methods marked for remove.
>
>> any existing method that's already deprecated, don't bother
>
> transitioning it to Duration.
>
> Fixed.
>
>> IllegalArgumentException... we should plan to mention this in the
>
> javadoc for those methods.
>
> Got it.
>
>> In Stores, windowSize and segmentInterval should also be durations.
>
> Fixed.
>
>> StreamsMetrics, recordLatency ... this one is better left alone.
>
> OK. I removed this method from KIP [1].
>
> Two more questions question about implementation:
>
> 1. We have serveral methods without parameters.
> In java we can't have two methods with parameters with the same name.
> It wouldn't compile.
> So we have to rename new methods. Please, see suggested names and share
> your thoughts:
>
> Windows {
> long size() -> Duration windowSize();
> }
>
> Window {
> long start() -> Instant startTime();
> long end() -> Instant endTime();
> }
>
> SessionWindows {
> long inactivityGap() -> Duration inactivityGapDuration();
> }
>
> TimeWindowedDeserializer {
> Long getWindowSize() -> Duration getWindowSizeDuration(); // or
>>>
>>> just
> Duration windowSize();
> }
>
> SessionBytesStoreSupplier {
> long retentionPeriod() -> Duration retentionPeriodDuration();
> }
>
> WindowBytesStoreSupplier {
> long windowSize() -> Duration windowSizeDuration();
> long retentionPeriod() -> Duration retentionPeriodDuration();
> }
>
> 2. Do we want to use Duration and Instant inside API implementations?
>
> IGNITE-7277: "Durations 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Dear, commiters.

Please, pay attention to this KIP and share your opinion.

В Вт, 21/08/2018 в 11:14 -0500, John Roesler пишет:
> I'll solicit more reviews. Let's get at least one committer to chime in
> before we start a vote (since we need their approval anyway).
> -John
> 
> On Mon, Aug 20, 2018 at 12:39 PM Nikolay Izhikov 
> wrote:
> 
> > Hello, Ted.
> > 
> > Thanks for the comment.
> > 
> > I've edit KIP and change proposal to `windowSize`.
> > 
> > Guys, any other comments?
> > 
> > 
> > В Вс, 19/08/2018 в 14:57 -0700, Ted Yu пишет:
> > > bq. // or just Duration windowSize();
> > > 
> > > +1 to the above choice.
> > > The duration is obvious from the return type. For getter methods, we
> > 
> > don't
> > > use get as prefix (as least for new code).
> > > 
> > > Cheers
> > > 
> > > On Sun, Aug 19, 2018 at 8:03 AM Nikolay Izhikov 
> > 
> > wrote:
> > > 
> > > > Hello, John.
> > > > 
> > > > Thank you very much for your feedback!
> > > > I've addressed all your comments.
> > > > Please, see my answers and let my know is anything in KIP [1] needs to
> > 
> > be
> > > > improved.
> > > > 
> > > > > The correct choice is actually "Instant", not> "LocalDateTime"
> > > > 
> > > > I've changed the methods proposed in KIP [1] to use Instant.
> > > > 
> > > > > I noticed some recent APIs are> missing (see KIP-328)
> > > > > those APIs were just added and have never been released... you can
> > 
> > just
> > > > 
> > > > replace them.
> > > > 
> > > > I've added new methods to KIP [1].
> > > > Not released methods marked for remove.
> > > > 
> > > > > any existing method that's already deprecated, don't bother
> > > > 
> > > > transitioning it to Duration.
> > > > 
> > > > Fixed.
> > > > 
> > > > > IllegalArgumentException... we should plan to mention this in the
> > > > 
> > > > javadoc for those methods.
> > > > 
> > > > Got it.
> > > > 
> > > > > In Stores, windowSize and segmentInterval should also be durations.
> > > > 
> > > > Fixed.
> > > > 
> > > > > StreamsMetrics, recordLatency ... this one is better left alone.
> > > > 
> > > > OK. I removed this method from KIP [1].
> > > > 
> > > > Two more questions question about implementation:
> > > > 
> > > > 1. We have serveral methods without parameters.
> > > > In java we can't have two methods with parameters with the same name.
> > > > It wouldn't compile.
> > > > So we have to rename new methods. Please, see suggested names and share
> > > > your thoughts:
> > > > 
> > > > Windows {
> > > > long size() -> Duration windowSize();
> > > > }
> > > > 
> > > > Window {
> > > > long start() -> Instant startTime();
> > > > long end() -> Instant endTime();
> > > > }
> > > > 
> > > > SessionWindows {
> > > > long inactivityGap() -> Duration inactivityGapDuration();
> > > > }
> > > > 
> > > > TimeWindowedDeserializer {
> > > > Long getWindowSize() -> Duration getWindowSizeDuration(); // or
> > 
> > just
> > > > Duration windowSize();
> > > > }
> > > > 
> > > > SessionBytesStoreSupplier {
> > > > long retentionPeriod() -> Duration retentionPeriodDuration();
> > > > }
> > > > 
> > > > WindowBytesStoreSupplier {
> > > > long windowSize() -> Duration windowSizeDuration();
> > > > long retentionPeriod() -> Duration retentionPeriodDuration();
> > > > }
> > > > 
> > > > 2. Do we want to use Duration and Instant inside API implementations?
> > > > 
> > > > IGNITE-7277: "Durations potentially worsen memory pressure and gc
> > > > performance, so internally, we will still use longMs as the
> > 
> > representation."
> > > > IGNITE-7222: Duration used to store retention.
> > > > 
> > > > [1]
> > > > 
> > 
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
> > > > [2]
> > > > 
> > 
> > https://github.com/apache/kafka/commit/b3771ba22acad7870e38ff7f58820c5b50946787#diff-47289575d3e3e2449f27b3a7b6788e1aR64
> > > > 
> > > > В Пт, 17/08/2018 в 14:46 -0500, John Roesler пишет:
> > > > > Hi Nikolay,
> > > > > 
> > > > > Thanks for this very nice KIP!
> > > > > 
> > > > > To answer your questions:
> > > > > 1. Correct, we should not delete existing methods that have been
> > > > 
> > > > released,
> > > > > but ...
> > > > > 
> > > > > 2. Yes, we should deprecate the 'long' variants so that we can drop
> > 
> > them
> > > > > later on. Personally, I like to mention which version deprecated the
> > > > 
> > > > method
> > > > > so everyone can see later on how long it's been deprecated, but this
> > 
> > may
> > > > 
> > > > be
> > > > > controversial, so let's let other weigh in.
> > > > > 
> > > > > 3. I think you're asking whether it's appropriate to drop the "Ms"
> > > > 
> > > > suffix,
> > > > > and I think yes. So "long inactivityGapMs" would become "Duration
> > > > > inactivityGap".
> > > > > In the places where the parameter's name is just "duration", I think
> > 
> > we
> > > > 
> > > > can
> > > > > pick something more descriptive (I realize it was already
> > 
> > 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[John Roesler]

I'll solicit more reviews. Let's get at least one committer to chime in
before we start a vote (since we need their approval anyway).
-John

On Mon, Aug 20, 2018 at 12:39 PM Nikolay Izhikov 
wrote:

> Hello, Ted.
>
> Thanks for the comment.
>
> I've edit KIP and change proposal to `windowSize`.
>
> Guys, any other comments?
>
>
> В Вс, 19/08/2018 в 14:57 -0700, Ted Yu пишет:
> > bq. // or just Duration windowSize();
> >
> > +1 to the above choice.
> > The duration is obvious from the return type. For getter methods, we
> don't
> > use get as prefix (as least for new code).
> >
> > Cheers
> >
> > On Sun, Aug 19, 2018 at 8:03 AM Nikolay Izhikov 
> wrote:
> >
> > > Hello, John.
> > >
> > > Thank you very much for your feedback!
> > > I've addressed all your comments.
> > > Please, see my answers and let my know is anything in KIP [1] needs to
> be
> > > improved.
> > >
> > > > The correct choice is actually "Instant", not> "LocalDateTime"
> > >
> > > I've changed the methods proposed in KIP [1] to use Instant.
> > >
> > > > I noticed some recent APIs are> missing (see KIP-328)
> > > > those APIs were just added and have never been released... you can
> just
> > >
> > > replace them.
> > >
> > > I've added new methods to KIP [1].
> > > Not released methods marked for remove.
> > >
> > > > any existing method that's already deprecated, don't bother
> > >
> > > transitioning it to Duration.
> > >
> > > Fixed.
> > >
> > > > IllegalArgumentException... we should plan to mention this in the
> > >
> > > javadoc for those methods.
> > >
> > > Got it.
> > >
> > > > In Stores, windowSize and segmentInterval should also be durations.
> > >
> > > Fixed.
> > >
> > > > StreamsMetrics, recordLatency ... this one is better left alone.
> > >
> > > OK. I removed this method from KIP [1].
> > >
> > > Two more questions question about implementation:
> > >
> > > 1. We have serveral methods without parameters.
> > > In java we can't have two methods with parameters with the same name.
> > > It wouldn't compile.
> > > So we have to rename new methods. Please, see suggested names and share
> > > your thoughts:
> > >
> > > Windows {
> > > long size() -> Duration windowSize();
> > > }
> > >
> > > Window {
> > > long start() -> Instant startTime();
> > > long end() -> Instant endTime();
> > > }
> > >
> > > SessionWindows {
> > > long inactivityGap() -> Duration inactivityGapDuration();
> > > }
> > >
> > > TimeWindowedDeserializer {
> > > Long getWindowSize() -> Duration getWindowSizeDuration(); // or
> just
> > > Duration windowSize();
> > > }
> > >
> > > SessionBytesStoreSupplier {
> > > long retentionPeriod() -> Duration retentionPeriodDuration();
> > > }
> > >
> > > WindowBytesStoreSupplier {
> > > long windowSize() -> Duration windowSizeDuration();
> > > long retentionPeriod() -> Duration retentionPeriodDuration();
> > > }
> > >
> > > 2. Do we want to use Duration and Instant inside API implementations?
> > >
> > > IGNITE-7277: "Durations potentially worsen memory pressure and gc
> > > performance, so internally, we will still use longMs as the
> representation."
> > > IGNITE-7222: Duration used to store retention.
> > >
> > > [1]
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
> > > [2]
> > >
> https://github.com/apache/kafka/commit/b3771ba22acad7870e38ff7f58820c5b50946787#diff-47289575d3e3e2449f27b3a7b6788e1aR64
> > >
> > > В Пт, 17/08/2018 в 14:46 -0500, John Roesler пишет:
> > > > Hi Nikolay,
> > > >
> > > > Thanks for this very nice KIP!
> > > >
> > > > To answer your questions:
> > > > 1. Correct, we should not delete existing methods that have been
> > >
> > > released,
> > > > but ...
> > > >
> > > > 2. Yes, we should deprecate the 'long' variants so that we can drop
> them
> > > > later on. Personally, I like to mention which version deprecated the
> > >
> > > method
> > > > so everyone can see later on how long it's been deprecated, but this
> may
> > >
> > > be
> > > > controversial, so let's let other weigh in.
> > > >
> > > > 3. I think you're asking whether it's appropriate to drop the "Ms"
> > >
> > > suffix,
> > > > and I think yes. So "long inactivityGapMs" would become "Duration
> > > > inactivityGap".
> > > > In the places where the parameter's name is just "duration", I think
> we
> > >
> > > can
> > > > pick something more descriptive (I realize it was already
> "durationMs";
> > > > this is just a good time to improve it).
> > > > Also, you're correct that we shouldn't use a Duration to represent a
> > >
> > > moment
> > > > in time, like "startTime". The correct choice is actually "Instant",
> not
> > > > "LocalDateTime", though.
> > > >
> > >
> > >
> https://stackoverflow.com/questions/32437550/whats-the-difference-between-instant-and-localdatetime
> > > > explains why.
> > > >
> > > > I also had a few notes on the KIP itself:
> > > > 4. You might want to pull trunk again. I noticed some recent 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Ted.

Thanks for the comment.

I've edit KIP and change proposal to `windowSize`.

Guys, any other comments?


В Вс, 19/08/2018 в 14:57 -0700, Ted Yu пишет:
> bq. // or just Duration windowSize();
> 
> +1 to the above choice.
> The duration is obvious from the return type. For getter methods, we don't
> use get as prefix (as least for new code).
> 
> Cheers
> 
> On Sun, Aug 19, 2018 at 8:03 AM Nikolay Izhikov  wrote:
> 
> > Hello, John.
> > 
> > Thank you very much for your feedback!
> > I've addressed all your comments.
> > Please, see my answers and let my know is anything in KIP [1] needs to be
> > improved.
> > 
> > > The correct choice is actually "Instant", not> "LocalDateTime"
> > 
> > I've changed the methods proposed in KIP [1] to use Instant.
> > 
> > > I noticed some recent APIs are> missing (see KIP-328)
> > > those APIs were just added and have never been released... you can just
> > 
> > replace them.
> > 
> > I've added new methods to KIP [1].
> > Not released methods marked for remove.
> > 
> > > any existing method that's already deprecated, don't bother
> > 
> > transitioning it to Duration.
> > 
> > Fixed.
> > 
> > > IllegalArgumentException... we should plan to mention this in the
> > 
> > javadoc for those methods.
> > 
> > Got it.
> > 
> > > In Stores, windowSize and segmentInterval should also be durations.
> > 
> > Fixed.
> > 
> > > StreamsMetrics, recordLatency ... this one is better left alone.
> > 
> > OK. I removed this method from KIP [1].
> > 
> > Two more questions question about implementation:
> > 
> > 1. We have serveral methods without parameters.
> > In java we can't have two methods with parameters with the same name.
> > It wouldn't compile.
> > So we have to rename new methods. Please, see suggested names and share
> > your thoughts:
> > 
> > Windows {
> > long size() -> Duration windowSize();
> > }
> > 
> > Window {
> > long start() -> Instant startTime();
> > long end() -> Instant endTime();
> > }
> > 
> > SessionWindows {
> > long inactivityGap() -> Duration inactivityGapDuration();
> > }
> > 
> > TimeWindowedDeserializer {
> > Long getWindowSize() -> Duration getWindowSizeDuration(); // or just
> > Duration windowSize();
> > }
> > 
> > SessionBytesStoreSupplier {
> > long retentionPeriod() -> Duration retentionPeriodDuration();
> > }
> > 
> > WindowBytesStoreSupplier {
> > long windowSize() -> Duration windowSizeDuration();
> > long retentionPeriod() -> Duration retentionPeriodDuration();
> > }
> > 
> > 2. Do we want to use Duration and Instant inside API implementations?
> > 
> > IGNITE-7277: "Durations potentially worsen memory pressure and gc
> > performance, so internally, we will still use longMs as the representation."
> > IGNITE-7222: Duration used to store retention.
> > 
> > [1]
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
> > [2]
> > https://github.com/apache/kafka/commit/b3771ba22acad7870e38ff7f58820c5b50946787#diff-47289575d3e3e2449f27b3a7b6788e1aR64
> > 
> > В Пт, 17/08/2018 в 14:46 -0500, John Roesler пишет:
> > > Hi Nikolay,
> > > 
> > > Thanks for this very nice KIP!
> > > 
> > > To answer your questions:
> > > 1. Correct, we should not delete existing methods that have been
> > 
> > released,
> > > but ...
> > > 
> > > 2. Yes, we should deprecate the 'long' variants so that we can drop them
> > > later on. Personally, I like to mention which version deprecated the
> > 
> > method
> > > so everyone can see later on how long it's been deprecated, but this may
> > 
> > be
> > > controversial, so let's let other weigh in.
> > > 
> > > 3. I think you're asking whether it's appropriate to drop the "Ms"
> > 
> > suffix,
> > > and I think yes. So "long inactivityGapMs" would become "Duration
> > > inactivityGap".
> > > In the places where the parameter's name is just "duration", I think we
> > 
> > can
> > > pick something more descriptive (I realize it was already "durationMs";
> > > this is just a good time to improve it).
> > > Also, you're correct that we shouldn't use a Duration to represent a
> > 
> > moment
> > > in time, like "startTime". The correct choice is actually "Instant", not
> > > "LocalDateTime", though.
> > > 
> > 
> > https://stackoverflow.com/questions/32437550/whats-the-difference-between-instant-and-localdatetime
> > > explains why.
> > > 
> > > I also had a few notes on the KIP itself:
> > > 4. You might want to pull trunk again. I noticed some recent APIs are
> > > missing (see KIP-328).
> > > 
> > > 5. Speaking of KIP-328: those APIs were just added and have never been
> > > released, so there's no need to deprecate the methods, you can just
> > 
> > replace
> > > them.
> > > 
> > > 6. For any existing method that's already deprecated, don't bother
> > > transitioning it to Duration. I think the examples I noticed were
> > > deprecated in KIP-328, so you'll see what I'm talking about when you pull
> > > trunk 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Ted Yu]

bq. // or just Duration windowSize();

+1 to the above choice.
The duration is obvious from the return type. For getter methods, we don't
use get as prefix (as least for new code).

Cheers

On Sun, Aug 19, 2018 at 8:03 AM Nikolay Izhikov  wrote:

> Hello, John.
>
> Thank you very much for your feedback!
> I've addressed all your comments.
> Please, see my answers and let my know is anything in KIP [1] needs to be
> improved.
>
> > The correct choice is actually "Instant", not> "LocalDateTime"
>
> I've changed the methods proposed in KIP [1] to use Instant.
>
> > I noticed some recent APIs are> missing (see KIP-328)
> > those APIs were just added and have never been released... you can just
> replace them.
>
> I've added new methods to KIP [1].
> Not released methods marked for remove.
>
> > any existing method that's already deprecated, don't bother
> transitioning it to Duration.
>
> Fixed.
>
> > IllegalArgumentException... we should plan to mention this in the
> javadoc for those methods.
>
> Got it.
>
> > In Stores, windowSize and segmentInterval should also be durations.
>
> Fixed.
>
> > StreamsMetrics, recordLatency ... this one is better left alone.
>
> OK. I removed this method from KIP [1].
>
> Two more questions question about implementation:
>
> 1. We have serveral methods without parameters.
> In java we can't have two methods with parameters with the same name.
> It wouldn't compile.
> So we have to rename new methods. Please, see suggested names and share
> your thoughts:
>
> Windows {
> long size() -> Duration windowSize();
> }
>
> Window {
> long start() -> Instant startTime();
> long end() -> Instant endTime();
> }
>
> SessionWindows {
> long inactivityGap() -> Duration inactivityGapDuration();
> }
>
> TimeWindowedDeserializer {
> Long getWindowSize() -> Duration getWindowSizeDuration(); // or just
> Duration windowSize();
> }
>
> SessionBytesStoreSupplier {
> long retentionPeriod() -> Duration retentionPeriodDuration();
> }
>
> WindowBytesStoreSupplier {
> long windowSize() -> Duration windowSizeDuration();
> long retentionPeriod() -> Duration retentionPeriodDuration();
> }
>
> 2. Do we want to use Duration and Instant inside API implementations?
>
> IGNITE-7277: "Durations potentially worsen memory pressure and gc
> performance, so internally, we will still use longMs as the representation."
> IGNITE-7222: Duration used to store retention.
>
> [1]
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
> [2]
> https://github.com/apache/kafka/commit/b3771ba22acad7870e38ff7f58820c5b50946787#diff-47289575d3e3e2449f27b3a7b6788e1aR64
>
> В Пт, 17/08/2018 в 14:46 -0500, John Roesler пишет:
> > Hi Nikolay,
> >
> > Thanks for this very nice KIP!
> >
> > To answer your questions:
> > 1. Correct, we should not delete existing methods that have been
> released,
> > but ...
> >
> > 2. Yes, we should deprecate the 'long' variants so that we can drop them
> > later on. Personally, I like to mention which version deprecated the
> method
> > so everyone can see later on how long it's been deprecated, but this may
> be
> > controversial, so let's let other weigh in.
> >
> > 3. I think you're asking whether it's appropriate to drop the "Ms"
> suffix,
> > and I think yes. So "long inactivityGapMs" would become "Duration
> > inactivityGap".
> > In the places where the parameter's name is just "duration", I think we
> can
> > pick something more descriptive (I realize it was already "durationMs";
> > this is just a good time to improve it).
> > Also, you're correct that we shouldn't use a Duration to represent a
> moment
> > in time, like "startTime". The correct choice is actually "Instant", not
> > "LocalDateTime", though.
> >
> https://stackoverflow.com/questions/32437550/whats-the-difference-between-instant-and-localdatetime
> > explains why.
> >
> > I also had a few notes on the KIP itself:
> > 4. You might want to pull trunk again. I noticed some recent APIs are
> > missing (see KIP-328).
> >
> > 5. Speaking of KIP-328: those APIs were just added and have never been
> > released, so there's no need to deprecate the methods, you can just
> replace
> > them.
> >
> > 6. For any existing method that's already deprecated, don't bother
> > transitioning it to Duration. I think the examples I noticed were
> > deprecated in KIP-328, so you'll see what I'm talking about when you pull
> > trunk again.
> >
> > 7. Any method taking a Duration argument may throw an
> > IllegalArgumentException (we choose to convert ArithmeticException to
> > IllegalArgumentException, as I mentioned in the Jira ticket). We don't
> need
> > a "throws" declaration, but we should plan to mention this in the javadoc
> > for those methods.
> >
> > 8. In Stores, windowSize and segmentInterval should also be durations.
> >
> > 9. In StreamsMetrics, recordLatency could be just a Duration, but I
> > actually think this one is better left alone. 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, John.

Thank you very much for your feedback!
I've addressed all your comments.
Please, see my answers and let my know is anything in KIP [1] needs to be 
improved.

> The correct choice is actually "Instant", not> "LocalDateTime"

I've changed the methods proposed in KIP [1] to use Instant.

> I noticed some recent APIs are> missing (see KIP-328)
> those APIs were just added and have never been released... you can just 
> replace them.

I've added new methods to KIP [1].
Not released methods marked for remove.

> any existing method that's already deprecated, don't bother transitioning it 
> to Duration.

Fixed.

> IllegalArgumentException... we should plan to mention this in the javadoc for 
> those methods.

Got it.

> In Stores, windowSize and segmentInterval should also be durations.

Fixed.

> StreamsMetrics, recordLatency ... this one is better left alone.

OK. I removed this method from KIP [1].

Two more questions question about implementation: 

1. We have serveral methods without parameters. 
In java we can't have two methods with parameters with the same name.
It wouldn't compile. 
So we have to rename new methods. Please, see suggested names and share your 
thoughts:

Windows {
long size() -> Duration windowSize();
}

Window {
long start() -> Instant startTime();
long end() -> Instant endTime();
}

SessionWindows {
long inactivityGap() -> Duration inactivityGapDuration();
}

TimeWindowedDeserializer {
Long getWindowSize() -> Duration getWindowSizeDuration(); // or just 
Duration windowSize();
}

SessionBytesStoreSupplier {
long retentionPeriod() -> Duration retentionPeriodDuration();
}

WindowBytesStoreSupplier {
long windowSize() -> Duration windowSizeDuration();
long retentionPeriod() -> Duration retentionPeriodDuration();
}

2. Do we want to use Duration and Instant inside API implementations?

IGNITE-7277: "Durations potentially worsen memory pressure and gc performance, 
so internally, we will still use longMs as the representation."
IGNITE-7222: Duration used to store retention.

[1] 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
[2] 
https://github.com/apache/kafka/commit/b3771ba22acad7870e38ff7f58820c5b50946787#diff-47289575d3e3e2449f27b3a7b6788e1aR64

В Пт, 17/08/2018 в 14:46 -0500, John Roesler пишет:
> Hi Nikolay,
> 
> Thanks for this very nice KIP!
> 
> To answer your questions:
> 1. Correct, we should not delete existing methods that have been released,
> but ...
> 
> 2. Yes, we should deprecate the 'long' variants so that we can drop them
> later on. Personally, I like to mention which version deprecated the method
> so everyone can see later on how long it's been deprecated, but this may be
> controversial, so let's let other weigh in.
> 
> 3. I think you're asking whether it's appropriate to drop the "Ms" suffix,
> and I think yes. So "long inactivityGapMs" would become "Duration
> inactivityGap".
> In the places where the parameter's name is just "duration", I think we can
> pick something more descriptive (I realize it was already "durationMs";
> this is just a good time to improve it).
> Also, you're correct that we shouldn't use a Duration to represent a moment
> in time, like "startTime". The correct choice is actually "Instant", not
> "LocalDateTime", though.
> https://stackoverflow.com/questions/32437550/whats-the-difference-between-instant-and-localdatetime
> explains why.
> 
> I also had a few notes on the KIP itself:
> 4. You might want to pull trunk again. I noticed some recent APIs are
> missing (see KIP-328).
> 
> 5. Speaking of KIP-328: those APIs were just added and have never been
> released, so there's no need to deprecate the methods, you can just replace
> them.
> 
> 6. For any existing method that's already deprecated, don't bother
> transitioning it to Duration. I think the examples I noticed were
> deprecated in KIP-328, so you'll see what I'm talking about when you pull
> trunk again.
> 
> 7. Any method taking a Duration argument may throw an
> IllegalArgumentException (we choose to convert ArithmeticException to
> IllegalArgumentException, as I mentioned in the Jira ticket). We don't need
> a "throws" declaration, but we should plan to mention this in the javadoc
> for those methods.
> 
> 8. In Stores, windowSize and segmentInterval should also be durations.
> 
> 9. In StreamsMetrics, recordLatency could be just a Duration, but I
> actually think this one is better left alone. IMO, it's more effort for
> little gain to require users to construct a Duration before they call the
> method, since they vary likely call System.currentTimeNanos before and
> after the code in question.
> 
> These are quite a few notes, but they're all minor. Overall the KIP looks
> really good to me. Thanks for picking this up!
> -John
> 
> On Thu, Aug 16, 2018 at 9:55 AM Nikolay Izhikov  wrote:
> 
> > Hello, Kafka developers.
> > 
> > I would like to start a 

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[John Roesler]

Hi Nikolay,

Thanks for this very nice KIP!

To answer your questions:
1. Correct, we should not delete existing methods that have been released,
but ...

2. Yes, we should deprecate the 'long' variants so that we can drop them
later on. Personally, I like to mention which version deprecated the method
so everyone can see later on how long it's been deprecated, but this may be
controversial, so let's let other weigh in.

3. I think you're asking whether it's appropriate to drop the "Ms" suffix,
and I think yes. So "long inactivityGapMs" would become "Duration
inactivityGap".
In the places where the parameter's name is just "duration", I think we can
pick something more descriptive (I realize it was already "durationMs";
this is just a good time to improve it).
Also, you're correct that we shouldn't use a Duration to represent a moment
in time, like "startTime". The correct choice is actually "Instant", not
"LocalDateTime", though.
https://stackoverflow.com/questions/32437550/whats-the-difference-between-instant-and-localdatetime
explains why.

I also had a few notes on the KIP itself:
4. You might want to pull trunk again. I noticed some recent APIs are
missing (see KIP-328).

5. Speaking of KIP-328: those APIs were just added and have never been
released, so there's no need to deprecate the methods, you can just replace
them.

6. For any existing method that's already deprecated, don't bother
transitioning it to Duration. I think the examples I noticed were
deprecated in KIP-328, so you'll see what I'm talking about when you pull
trunk again.

7. Any method taking a Duration argument may throw an
IllegalArgumentException (we choose to convert ArithmeticException to
IllegalArgumentException, as I mentioned in the Jira ticket). We don't need
a "throws" declaration, but we should plan to mention this in the javadoc
for those methods.

8. In Stores, windowSize and segmentInterval should also be durations.

9. In StreamsMetrics, recordLatency could be just a Duration, but I
actually think this one is better left alone. IMO, it's more effort for
little gain to require users to construct a Duration before they call the
method, since they vary likely call System.currentTimeNanos before and
after the code in question.

These are quite a few notes, but they're all minor. Overall the KIP looks
really good to me. Thanks for picking this up!
-John

On Thu, Aug 16, 2018 at 9:55 AM Nikolay Izhikov  wrote:

> Hello, Kafka developers.
>
> I would like to start a discussion of KIP-358 [1].
> It based on a ticket KAFKA-7277 [2].
>
> I crawled through Stream API and made my suggestions for API changes.
>
> I have several questions about changes.
> Please, share your comments:
>
> 1. I propose do not remove existing API methods with long ms parameters.
> Is it correct?
>
> 2. Should we mark existing methods as deprecated?
>
> 3. Suggested changes in ticket description are `long durationMs` to
> `Duration duration` and similar.
> I suggest to change 'long startTimeMs` to `LocalDateTime startTime` also.
> Should we do it?
>
> Please, note, it very first KIP for me, so tell me if I miss something
> obvious for experienced Kafka developers.
>
> [1]
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
> [2] https://issues.apache.org/jira/browse/KAFKA-7277

[DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Nikolay Izhikov]

Hello, Kafka developers.

I would like to start a discussion of KIP-358 [1].
It based on a ticket KAFKA-7277 [2].

I crawled through Stream API and made my suggestions for API changes.

I have several questions about changes.
Please, share your comments:

1. I propose do not remove existing API methods with long ms parameters. Is it 
correct?

2. Should we mark existing methods as deprecated?

3. Suggested changes in ticket description are `long durationMs` to `Duration 
duration` and similar.
I suggest to change 'long startTimeMs` to `LocalDateTime startTime` also.
Should we do it?

Please, note, it very first KIP for me, so tell me if I miss something obvious 
for experienced Kafka developers.

[1] 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
[2] https://issues.apache.org/jira/browse/KAFKA-7277

signature.asc
Description: This is a digitally signed message part