[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[Kostas Kloudas (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964862#comment-16964862
 ] 

Kostas Kloudas commented on FLINK-6214:
---

>From the discussion here it seems that there is an alternative way of 
>implementing the functionality proposed, without making any changes in the 
>codebase. In addition, there does not seem to be much interest neither in the 
>PR nor here.

Given the above, and the fact that the windowing code in Flink is already a bit 
complicated, I will close this issue.

Feel free to re-open it if you think that this is a valid addition.

> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: API / DataStream
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>Assignee: leesf
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392966#comment-16392966
 ] 

ASF GitHub Bot commented on FLINK-6214:
---

Github user StephanEwen commented on the issue:

https://github.com/apache/flink/pull/5376
  
Ah, one problem: What happens if the offset is larger (by absolute value) 
than the window length? Then the offset would still be negative with this 
change.


> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>Priority: Major
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392962#comment-16392962
 ] 

ASF GitHub Bot commented on FLINK-6214:
---

Github user StephanEwen commented on the issue:

https://github.com/apache/flink/pull/5376
  
If we decide that we want to support negative offsets, this looks like a 
good implementation.

@aljoscha what is your take, should we support negative offsets?


> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>Priority: Major
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[Jelmer Kuperus (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342761#comment-16342761
 ] 

Jelmer Kuperus commented on FLINK-6214:
---

Proposed pull request : https://github.com/apache/flink/pull/5376

> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>Priority: Major
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342760#comment-16342760
 ] 

ASF GitHub Bot commented on FLINK-6214:
---

GitHub user jelmerk opened a pull request:

https://github.com/apache/flink/pull/5376

[FLINK-6214] WindowAssigners do not allow negative offsets

## What is the purpose of the change

The javadoc of TumblingEventTimeWindows and TumblingProcessingTimeWindows 
suggest that it is possible to use negative offsets but in practice this is not 
supported. This patch remedies this situation

## Brief change log

- updated behavior of TumblingEventTimeWindows & 
TumblingProcessingTimeWindows

## Verifying this change

This change added tests and can be verified as follows:

- Added testWindowAssignmentWithNegativeOffset method in 
TumblingEventTimeWindowsTest & TumblingProcessingTimeWindowsTest

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): no
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: no
  - The serializers: no
  - The runtime per-record code paths (performance sensitive): no
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: no
  - The S3 file system connector: no

## Documentation

  - Does this pull request introduce a new feature? no
  - If yes, how is the feature documented? not applicable


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jelmerk/flink negative_offsets

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5376.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5376


commit 07f5577d67155c86af72dac85f7c96b4fea06456
Author: Jelmer Kuperus 
Date:   2018-01-28T22:22:10Z

FLINK-6214: allow negative offsets




> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>Priority: Major
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313293#comment-16313293
 ] 

Aljoscha Krettek commented on FLINK-6214:
-

Nope, no reason except no-one has gotten around to it yet.

> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[Jelmer Kuperus (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312927#comment-16312927
 ] 

Jelmer Kuperus commented on FLINK-6214:
---

Is there any reason not to do the automatic conversion ? This is mildly 
confusing

> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951199#comment-15951199
 ] 

Aljoscha Krettek commented on FLINK-6214:
-

We could add automatic conversion from negative offsets to the respective 
positive offsets. I don't think this is a major show stopper, though.

> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[Vladislav Pernin (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950531#comment-15950531
 ] 

Vladislav Pernin commented on FLINK-6214:
-

Let's take the example of a simple one day window, but that has to begin at 
00:00:00 and not at 01:00:00 and ends at 00:00:00 next day.
of(Time.days(1), Time.hours(-1)) would produce the expected window : J 00:00:00 
-> J+1 00:00:00
of(Time.days(1), Time.hours(23)) would also work but not as easy to think about.

> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947887#comment-15947887
 ] 

Aljoscha Krettek commented on FLINK-6214:
-

I think this might be "working as intended" and we need to fix the 
documentation and Javadoc. Limiting the offset to 0 <= offset <= size 
simplifies the implementation (if I'm not wrong). An offset of -8 can be 
achieved by "offsetting in the other direction": An offset of 24 hours equals a 
0 hours offset, an offset of 23 hours is the same as a -1 hour offset and so on.

> WindowAssigners do not allow negative offsets
> -
>
> Key: FLINK-6214
> URL: https://issues.apache.org/jira/browse/FLINK-6214
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.2.0
>Reporter: Timo Walther
>
> Both the website and the JavaDoc promotes 
> ".window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) For 
> example, in China you would have to specify an offset of Time.hours(-8)". But 
> both the sliding and tumbling event time assigners do not allow offset to be 
> negative.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)