[
https://issues.apache.org/jira/browse/FLINK-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534757#comment-16534757
]
ASF GitHub Bot commented on FLINK-9407:
---------------------------------------
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/6075
Hi, Thanks @StephanEwen . For a short checkpoint time that whether lead to
poor compression or not, I will give an actual production test under
compression situation and as an attachment attached to
[FLINK-9407](https://issues.apache.org/jira/browse/FLINK-9407) in the next few
days.
Actually, We having been using this PR (but not polished yet) in our
production environment for a long time. And getting a very nice results ending
that compared to spark streaming. I will put the test results in the form of
attachments to [FLINK-9407](https://issues.apache.org/jira/browse/FLINK-9407)
as a reference. And what you are more concerned about the short checkpoint
interval will lead to poor compression, yea I would agree but we need a test
for it. But we have all known that the longer snapshots interval, the lower
performance impact of asynchronous or synchronous snapshotting we would get.
So, I think people do not inclined to adopt short checkpoint intervals for
getting a high throughput and low latency in production env. For example, in
our calculation of uv jobs, the checkpoint intervals is 30 seconds. Anyway, I
will still give a test results under the compression situation.
I suggest this PR can merge as a temporary solution to reduce the user's
learning curve since some users already had used ```BucketingSink``` in their
project and wants this functionality as we can see in the user mail list a few
days ago. And In a short time, they may not switch to a new sink
```StreamingFileSink ``` . This may be one of the good reasons.
By the way, I will watch
[FLINK-9749](https://issues.apache.org/jira/browse/FLINK-9749) and the subtask
[FLINK-9753](https://issues.apache.org/jira/browse/FLINK-9753) soon and give
more response in the next couple of days.
> Support orc rolling sink writer
> -------------------------------
>
> Key: FLINK-9407
> URL: https://issues.apache.org/jira/browse/FLINK-9407
> Project: Flink
> Issue Type: New Feature
> Components: filesystem-connector
> Reporter: zhangminglei
> Assignee: zhangminglei
> Priority: Major
> Labels: patch-available, pull-request-available
>
> Currently, we only support {{StringWriter}}, {{SequenceFileWriter}} and
> {{AvroKeyValueSinkWriter}}. I would suggest add an orc writer for rolling
> sink.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
