[
https://issues.apache.org/jira/browse/FLINK-19345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201317#comment-17201317
]
Kostas Kloudas commented on FLINK-19345:
----------------------------------------
Hi [~lzljs3620320], as you may have noticed there is an ongoing discussion
about the new sink interface going on in the dev mailing list and the target is
to have it ready for the 1.12.
Instead of having parallel efforts, why not contributing to the discussion
there so that our efforts are aligned? Also opening a new discussion although
there is already an open one for similar problems, I think it slows down the
process of reaching a consensus.
I will have a look at the document attached here, but I would think that
participating in the discussion on the FLIP about the unified sinks would be a
lot more helpful also for the community.
> Introduce File streaming sink compaction
> ----------------------------------------
>
> Key: FLINK-19345
> URL: https://issues.apache.org/jira/browse/FLINK-19345
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Runtime
> Reporter: Jingsong Lee
> Assignee: Jingsong Lee
> Priority: Major
> Fix For: 1.12.0
>
>
> Users often complain that many small files are written out. Small files will
> affect the performance of file reading and the DFS system, and even the
> stability of the DFS system.
> Target:
> * Compact all files generated by this job in a single checkpoint.
> * With compaction, Users can have smaller checkpoint interval, even to
> seconds.
> Document:
> https://docs.google.com/document/d/1cdlyoqgBq9yJEiHFBziimIoKHapQiEY2-0Tn8IF6G-c/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
