[
https://issues.apache.org/jira/browse/FLINK-4419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439327#comment-15439327
]
Greg Hogan commented on FLINK-4419:
-----------------------------------
My apologies for being late to this discussion. Are we not able to recover when
a downstream operator fails if the spilled files are written to redundant
storage? That would not require changes to the DataSet API and would not reduce
performance.
CC'ing [~StephanEwen] and [~till.rohrmann]
> Batch improvement for supporting dfs as a ResultPartitionType
> -------------------------------------------------------------
>
> Key: FLINK-4419
> URL: https://issues.apache.org/jira/browse/FLINK-4419
> Project: Flink
> Issue Type: Improvement
> Components: Batch Connectors and Input/Output Formats
> Reporter: shuai.xu
> Assignee: shuai.xu
>
> This is the root issue to track a improvement for batch, which will enable
> dfs as a ResultPartitionType, so that upstream node can exist totally after
> finished and need not be restarted if downstream nodes fail.
> Full design is shown in
> (https://docs.google.com/document/d/15HtCtc9Gk8SyHsAezM7Od1opAHgnxLeHm3VX7A8fa-4/edit#).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
