[
https://issues.apache.org/jira/browse/FLINK-19109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Anderson updated FLINK-19109:
-----------------------------------
Description:
Attempting to generate watermarks chained to the Split Reader /
ContinuousFileReaderOperator, as in
{code:java}
SingleOutputStreamOperator<Event> results = env
.readTextFile(...)
.map(...)
.assignTimestampsAndWatermarks(bounded)
.keyBy(...)
.process(...);{code}
leads to the Watermarks failing to be produced. Breaking the chain, via
{{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using
punctuated watermarks also avoids the issue.
Looking at this in the debugger reveals that timer service is being prematurely
quiesced.
In many respects this is FLINK-7666 brought back to life.
The problem is not present in 1.9.3.
There's a minimal reproducible example in
[https://github.com/alpinegizmo/flink-question-001/tree/bug].
was:
Attempting to generate watermarks chained to the Split Reader /
ContinuousFileReaderOperator, as in
{{ SingleOutputStreamOperator<Event> results = env
.readTextFile(...)
.map(...)
.assignTimestampsAndWatermarks(bounded)
.keyBy(...)
.process(...);
}}
leads to the Watermarks failing to be produced. Breaking the chain, via
{{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using
punctuated watermarks also avoids the issue.
Looking at this in the debugger reveals that timer service is being prematurely
quiesced.
In many respects this is FLINK-7666 brought back to life.
The problem is not present in 1.9.3.
There's a minimal reproducible example in
https://github.com/alpinegizmo/flink-question-001/tree/bug.
> Split Reader eats chained periodic watermarks
> ---------------------------------------------
>
> Key: FLINK-19109
> URL: https://issues.apache.org/jira/browse/FLINK-19109
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.10.0, 1.10.1, 1.11.0, 1.10.2, 1.11.1
> Reporter: David Anderson
> Priority: Critical
>
> Attempting to generate watermarks chained to the Split Reader /
> ContinuousFileReaderOperator, as in
> {code:java}
> SingleOutputStreamOperator<Event> results = env
> .readTextFile(...)
> .map(...)
> .assignTimestampsAndWatermarks(bounded)
> .keyBy(...)
> .process(...);{code}
> leads to the Watermarks failing to be produced. Breaking the chain, via
> {{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using
> punctuated watermarks also avoids the issue.
> Looking at this in the debugger reveals that timer service is being
> prematurely quiesced.
> In many respects this is FLINK-7666 brought back to life.
> The problem is not present in 1.9.3.
> There's a minimal reproducible example in
> [https://github.com/alpinegizmo/flink-question-001/tree/bug].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
