[
https://issues.apache.org/jira/browse/FLINK-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108743#comment-15108743
]
ASF GitHub Bot commented on FLINK-3256:
---------------------------------------
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1526#issuecomment-173240931
Thanks for fixing this, looks like a good issue!
One suggestion for a change: Rather than collecting the CoLocationGroups in
an extra set in the ExecutionGraph, can you iterate over the
ExecutionJobvertices and simply reset them there?
I think each field less on the ExecutionGraph makes this simpler to
maintain and more future-proof.
> Invalid execution graph cleanup for jobs with colocation groups
> ---------------------------------------------------------------
>
> Key: FLINK-3256
> URL: https://issues.apache.org/jira/browse/FLINK-3256
> Project: Flink
> Issue Type: Bug
> Components: Distributed Runtime
> Reporter: Paris Carbone
> Assignee: Paris Carbone
> Priority: Blocker
>
> Currently, upon restarting an execution graph, we clean-up the colocation
> constraints for each group present in an ExecutionJobVertex respectively.
> This can lead to invalid reconfiguration upon a restart or any other activity
> that relies on state cleanup of the execution graph. For example, upon
> restarting a DataStream job with iterations the following steps are executed:
> 1) IterationSource colgroup constraints are reset
> 2) IterationSource execution vertices reset and create new colocation
> constraints
> 3) IterationSink colgroup constraints are reset
> 4) IterationSink execution vertices reset and create different colocation
> constraints.
> This can be trivially fixed by reseting colocation groups independently from
> ExecutionJobVertices, thus, updating them once per reconfiguration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
