Yun Gao created FLINK-17593:
-------------------------------
Summary: Support arbitrary recovery mechanism for PartFileWriter
Key: FLINK-17593
URL: https://issues.apache.org/jira/browse/FLINK-17593
Project: Flink
Issue Type: New Feature
Components: Connectors / FileSystem
Reporter: Yun Gao
Fix For: 1.11.0
Currently Bucket relies directly on _RecoverableOutputStream_ provided by
FileSystem to achieve snapshotting and recovery the in-progress part file for
all the PartFileWriter implementations. This would require that the
PartFileWriter must be based on the OutputStream.
To support the path-based PartFileWriter required by the Hive Sink, we will
first need to abstract the snapshotting mechanism of the PartFileWriter and
make RecoverableOutputStream to be one type of implementation, thus we could
decouple PartFileWriter with the output streams.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
