[
https://issues.apache.org/jira/browse/FLINK-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159544#comment-17159544
]
Lu Niu commented on FLINK-17627:
--------------------------------
Hi, [~lzljs3620320]. thanks for replying.
I took a look at {{StreamingFileWriter}} and {{StreamingFileCommitter}}. The
solution for writing _SUCCESS logic is to create another operator
\{StreamingFileCommitter} which receives commit messages sent form
{{StreamingFileWriter.}}
However, in the DataStream API, and there is no flexibility in
RichSinkFunction to create operator after a sink. Any suggestions? thanks
> Add support for writing _SUCCESS file with StreamingFileSink
> ------------------------------------------------------------
>
> Key: FLINK-17627
> URL: https://issues.apache.org/jira/browse/FLINK-17627
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Reporter: Robert Metzger
> Priority: Major
> Labels: usability
>
> (Note: This feature has been requested by multiple users:
> https://lists.apache.org/[email protected]:lte=10000M:_SUCCESS)
> Hadoop Map Reduce is writing a _SUCCESS file to output directories once the
> result has been completely written.
> Users migrating from Hadoop MR to Flink want to have a similar behavior in
> Flinks StreamingFileSink.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
