[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Rohit Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766088#comment-15766088
 ] 

Rohit Agarwal commented on FLINK-5247:
--

Hi [~aljoscha],

I updated the PR with what be discussed. Please take a look.

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Rohit Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750858#comment-15750858
 ] 

Rohit Agarwal commented on FLINK-5247:
--

Sorry, I am a bit busy these days. I will try to get this done during the 
weekend.

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741913#comment-15741913
 ] 

Aljoscha Krettek commented on FLINK-5247:
-

Hi [~ragarwal], any updates on this?

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727584#comment-15727584
 ] 

Aljoscha Krettek commented on FLINK-5247:
-

Changing the scope of this issue and then changing the PR would be good.

Thanks for working on this, again! :-)

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Rohit Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726660#comment-15726660
 ] 

Rohit Agarwal commented on FLINK-5247:
--

So, do you want me to update the PR to not throw the exception and make 
allowedLateness method a no-op for processing-time windows or is the current PR 
fine?

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724298#comment-15724298
 ] 

Aljoscha Krettek commented on FLINK-5247:
-

I see. Thanks for looking into this!

I actually have a fix for this (ignore allowedLateness for processing time) 
buried in this PR: 
https://github.com/apache/flink/pull/2572/files#diff-408a499e1a35840c52e29b7ccab866b1R525
 (You can look at WindowOperator.cleanupTime()).

I think it would be better to ignore setting an allowed lateness for processing 
time. But you're right that the checks should be fixed.

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Rohit Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723552#comment-15723552
 ] 

Rohit Agarwal commented on FLINK-5247:
--

The existing code already had provisions to throw exceptions when 
allowedLateness is set to non-zero value for processing time windows.
{code}
throw new IllegalArgumentException("Setting the allowed 
lateness is only valid for event-time windows.");
{code}
I created this JIRA because that existing code was not working as intended: 
https://github.com/apache/flink/pull/2929/files

Currently, if you set allowedLateness to non-zero value for processing-time 
windows, flink would update allowedLateness to that value. (This probably 
results in delays in window purging.)

We can make allowedLateness method a no-op for processing-time windows, I 
didn't do that because looking at the existing code that didn't seem to be the 
intention.

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15722172#comment-15722172
 ] 

Aljoscha Krettek commented on FLINK-5247:
-

I'm not sure we actually want this. The only reason for the existence of the 
"stream time characteristic" (settable on {{StreamExecutionEnvironment}}) is to 
be able to quickly switch a pipeline between processing time and event time. 
(Whether that's helpful beyond simple demos/explorations is a different 
discussion)

If we throw an exception instead of defaulting to zero internally we break this 
feature. Setting an allowed lateness other than zero for processing time 
windows is not per se wrong. It just cannot ever occur that we see elements so 
defaulting to zero internally is a runtime optimisation.

> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718679#comment-15718679
 ] 

ASF GitHub Bot commented on FLINK-5247:
---

Github user mindprince commented on the issue:

https://github.com/apache/flink/pull/2929
  
cc - @kl0u, @aljoscha for review


> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-5247) Setting allowedLateness to a non-zero value should throw exception for processing-time windows

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718675#comment-15718675
 ] 

ASF GitHub Bot commented on FLINK-5247:
---

GitHub user mindprince opened a pull request:

https://github.com/apache/flink/pull/2929

[FLINK-5247] Fix check to make sure that we throw error when allowed 
lateness is set for non event-time windows.

Also, fix outdated documentation.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mindprince/flink FLINK-5247-allowed-lateness

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/2929.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2929


commit 10d523865e07df61e5a1c2b35e2201b498c46426
Author: Rohit Agarwal 
Date:   2016-12-03T20:15:45Z

[FLINK-5247] Fix check to make sure that we throw error when allowed 
lateness is set for non event-time windows.

Also, fix outdated documentation.




> Setting allowedLateness to a non-zero value should throw exception for 
> processing-time windows
> --
>
> Key: FLINK-5247
> URL: https://issues.apache.org/jira/browse/FLINK-5247
> Project: Flink
>  Issue Type: Bug
>  Components: Streaming
>Affects Versions: 1.1.0, 1.1.1, 1.1.2, 1.1.3
>Reporter: Rohit Agarwal
>
> Related to FLINK-3714 and FLINK-4239



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)