[
https://issues.apache.org/jira/browse/FLINK-22950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361170#comment-17361170
]
Piotr Nowojski commented on FLINK-22950:
----------------------------------------
Yes, that's current expected behaviour. I've changed type of this issue to
improvement.
In short, the generic back-pressure mechanism used in Flink relies on the
presence/absence of the network `Buffer`s in the Task's buffer pool. If task is
not writing something to an external system like a Sink, there is no generic
way for Flink to distinguish backpressure from busy. To solve this, we would
need to first provide availability API to the Sinks. Next each sink would need
to implement it on its own. Many of the sinks would not be able to implement it
thought.
Part of this was being proposed and discussed in FLIP-143 [1][2] and more
recently in FLIP-171
[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-143-Unified-Sink-API-tp44602p44872.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-143-Unified-Sink-API-tp44602p44930.html
[3]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink#FLIP171:AsyncSink-SimplifiedAsyncSinkWriterforKinesisDataStreams
> Back-pressure report of final sink operator is not accurate
> -----------------------------------------------------------
>
> Key: FLINK-22950
> URL: https://issues.apache.org/jira/browse/FLINK-22950
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Metrics, Runtime / Task
> Affects Versions: 1.13.1
> Reporter: Fu Kai
> Priority: Major
>
> We noticed the back-pressure status of the final sink on the job graph is not
> accurate. It's apparently the final sink operator should receive
> back-pressure from the sink storage(ElasitcSearch/Kafka), while it's always
> showing not back-pressured(with value of 0, but always showing 100% busy
> though).
> This is confirmed by switching the sink to be more powerful and without
> changing any other configurations. We noticed the record publish rate
> increases significantly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
