Márton Balassi created FLINK-2154:
-------------------------------------
Summary: ActiveTriggerPolicy collects elements to a closed buffer
after job finishes
Key: FLINK-2154
URL: https://issues.apache.org/jira/browse/FLINK-2154
Project: Flink
Issue Type: Bug
Components: Streaming
Affects Versions: 0.9
Reporter: Márton Balassi
When gracefully finishing a time windowing job I have witnessed the following
exceptions. The thread triggering the active policy tries collecting the data,
even though the buffer pool has been already destroyed as the operator has
finished.
16:06:44,419 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- StreamDiscretizer -> SlidingTimeGroupedPreReducer -> (Filter, ExtractParts)
(1/4) (a28aa6212bd0c4eca271c133eb86a223) switched from RUNNING to FINISHED
16:06:44,417 INFO org.apache.flink.runtime.testingUtils.TestingTaskManager
- Unregistering task and sending final execution state FINISHED to JobManager
for task CoFlatMap -> Window Flatten (891e684ea2c1d5155b2363057035fca0)
16:06:44,419 INFO org.apache.flink.runtime.client.JobClient
- 06/03/2015 16:06:44 StreamDiscretizer -> SlidingTimeGroupedPreReducer ->
(Filter, ExtractParts)(1/4) switched to FINISHED
16:06:44,419 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- StreamDiscretizer -> SlidingTimeGroupedPreReducer -> (Filter, ExtractParts)
(3/4) (2e22e0ba633eb1319adc9b48ed7ff477) switched from RUNNING to FINISHED
16:06:44,420 INFO org.apache.flink.runtime.client.JobClient
- 06/03/2015 16:06:44 StreamDiscretizer -> SlidingTimeGroupedPreReducer ->
(Filter, ExtractParts)(3/4) switched to FINISHED
16:06:44,420 INFO org.apache.flink.runtime.testingUtils.TestingTaskManager
- Unregistering task and sending final execution state FINISHED to JobManager
for task CoFlatMap -> Window Flatten (dadf928e00ab1b6f9685bf08e8d447d8)
16:06:44,421 INFO org.apache.flink.runtime.testingUtils.TestingTaskManager
- Unregistering task and sending final execution state FINISHED to JobManager
for task CoFlatMap -> Window Flatten (e211fc565e81fbdaeda0c20702c83fab)
16:06:44,421 INFO org.apache.flink.runtime.testingUtils.TestingTaskManager
- Unregistering task and sending final execution state FINISHED to JobManager
for task CoFlatMap -> Window Flatten (e92b325076707068b32780d30a3355a9)
16:06:44,422 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- CoFlatMap -> Window Flatten (2/4) (891e684ea2c1d5155b2363057035fca0)
switched from RUNNING to FINISHED
16:06:44,422 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- CoFlatMap -> Window Flatten (1/4) (dadf928e00ab1b6f9685bf08e8d447d8)
switched from RUNNING to FINISHED
16:06:44,422 INFO org.apache.flink.runtime.client.JobClient
- 06/03/2015 16:06:44 CoFlatMap -> Window Flatten(2/4) switched to FINISHED
16:06:44,423 INFO org.apache.flink.runtime.client.JobClient
- 06/03/2015 16:06:44 CoFlatMap -> Window Flatten(1/4) switched to FINISHED
16:06:44,423 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- CoFlatMap -> Window Flatten (3/4) (e211fc565e81fbdaeda0c20702c83fab)
switched from RUNNING to FINISHED
16:06:44,424 INFO org.apache.flink.runtime.client.JobClient
- 06/03/2015 16:06:44 CoFlatMap -> Window Flatten(3/4) switched to FINISHED
16:06:44,424 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- CoFlatMap -> Window Flatten (4/4) (e92b325076707068b32780d30a3355a9)
switched from RUNNING to FINISHED
16:06:44,424 INFO org.apache.flink.runtime.client.JobClient
- 06/03/2015 16:06:44 CoFlatMap -> Window Flatten(4/4) switched to FINISHED
16:06:49,151 ERROR org.apache.flink.streaming.api.collector.StreamOutput
- Emit failed due to: java.lang.IllegalStateException: Buffer pool is
destroyed.
at
org.apache.flink.runtime.io.network.buffer.LocalBufferPool.requestBuffer(LocalBufferPool.java:144)
at
org.apache.flink.runtime.io.network.buffer.LocalBufferPool.requestBufferBlocking(LocalBufferPool.java:133)
at
org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:92)
at
org.apache.flink.streaming.runtime.io.StreamRecordWriter.emit(StreamRecordWriter.java:58)
at
org.apache.flink.streaming.api.collector.StreamOutput.collect(StreamOutput.java:62)
at
org.apache.flink.streaming.api.collector.CollectorWrapper.collect(CollectorWrapper.java:40)
at
org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:34)
at
org.apache.flink.streaming.runtime.tasks.OutputHandler$OperatorCollector.collect(OutputHandler.java:232)
at
org.apache.flink.streaming.api.collector.CollectorWrapper.collect(CollectorWrapper.java:40)
at
org.apache.flink.streaming.api.windowing.windowbuffer.SlidingPreReducer.emitWindow(SlidingPreReducer.java:57)
at
org.apache.flink.streaming.api.operators.windowing.StreamWindowBuffer.handleWindowEvent(StreamWindowBuffer.java:54)
at
org.apache.flink.streaming.api.operators.windowing.StreamWindowBuffer.handleWindowEvent(StreamWindowBuffer.java:59)
at
org.apache.flink.streaming.api.operators.windowing.StreamWindowBuffer.processElement(StreamWindowBuffer.java:44)
at
org.apache.flink.streaming.api.operators.windowing.StreamWindowBuffer.processElement(StreamWindowBuffer.java:29)
at
org.apache.flink.streaming.runtime.tasks.OutputHandler$OperatorCollector.collect(OutputHandler.java:232)
at
org.apache.flink.streaming.api.collector.CollectorWrapper.collect(CollectorWrapper.java:40)
at
org.apache.flink.streaming.api.operators.windowing.StreamDiscretizer.emitWindow(StreamDiscretizer.java:133)
at
org.apache.flink.streaming.api.operators.windowing.StreamDiscretizer.triggerOnFakeElement(StreamDiscretizer.java:121)
at
org.apache.flink.streaming.api.operators.windowing.StreamDiscretizer$WindowingCallback.sendFakeElement(StreamDiscretizer.java:194)
at
org.apache.flink.streaming.api.windowing.policy.TimeTriggerPolicy.activeFakeElementEmission(TimeTriggerPolicy.java:117)
at
org.apache.flink.streaming.api.windowing.policy.TimeTriggerPolicy$TimeCheck.run(TimeTriggerPolicy.java:144)
at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
