[
https://issues.apache.org/jira/browse/FLINK-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719746#comment-16719746
]
zhijiang commented on FLINK-11035:
----------------------------------
[~pnowojski], thanks for your explanation! I am just not sure whether you have
other thoughts for this issue before, and I am clear now.
If there triggers a flush between two {{BufferBuilder}}'s add operation, then
we can get benefits from current design. In other words the flush thread help
us do the notification work to some extent. But the flush timeout is
configured, not very determined, especially for batch job no flush mechanism.
So it has two sides, and the disadvantage aspect may be just a corner case.
I have not found the problems in real world yet, only when we mock to build
such special scenarios. So this improvement is not critical and determined now,
we can keep the current logic until we have new finds. :)
> Notify data available to network stack immediately after finishing
> BufferBuilder
> --------------------------------------------------------------------------------
>
> Key: FLINK-11035
> URL: https://issues.apache.org/jira/browse/FLINK-11035
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.8.0
> Reporter: zhijiang
> Assignee: zhijiang
> Priority: Minor
>
> The data availability notification for network relies on whether there are
> finished _BufferBuilder_ or flush triggered. If flush is not triggered and
> the first _BufferBuilder_ enqueues into the subpartition, although this
> _BufferBuilder_ is finished on _RecordWriter_ side, it has to rely on
> enqueuing the second _BufferBuilder_ to trigger notification available. It
> may bring some delays for transporting the finished _BufferBuilder_ in
> network, especially there has a blocking operation for requesting the second
> _BufferBuilder_ from pool.
> Supposing there is only one available buffer in LocalBufferPool in extreme
> scenarios, if the first _BufferBuilder_ is not transported and recycled, the
> requesting for second _BufferBuilder_ will be blocked all the time.
> I propose to add a _notifyBufferFinished_ method in _ResultPartitionWriter_
> interface, then _RecordWriter_ can notify via it after _BufferBuilder_
> finished_._
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
