[
https://issues.apache.org/jira/browse/FLINK-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374090#comment-16374090
]
ASF GitHub Bot commented on FLINK-7428:
---------------------------------------
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
With the recent update of the netty version in flink-shaded (see
https://github.com/apache/flink-shaded/commit/1233f1bb0e2b9fafa4260603aa130b7eb9995a7a),
most of this PR is indirectly included. However, I probably need to fall back
to the original code creating a copy at least for the fallback non-credit based
paths due to the original problem.
Additionally, since the introduction of the credit-based flow control we
should not use too much memory on the receiver side anymore: memory size in the
range of once in our buffers, at most one (or two depending on the above) more
copies in netty temporarily.
Ultimately, we could go one step further of decoding directly to our
buffers here, but that is not covered by this PR and will only work with
credit-based flow control.
> avoid one additional buffer copy when receiving messages
> --------------------------------------------------------
>
> Key: FLINK-7428
> URL: https://issues.apache.org/jira/browse/FLINK-7428
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.4.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Major
>
> By using {{LengthFieldBasedFrameDecoder}}, we create one unnecessary (netty)
> buffer copy in this class which could be easily avoided since we can ensure
> that the buffer is free to be released after decoding it in the
> {{NettyMessageDecoder}} and into our own buffer and/or events.
> The solution would be to make {{NettyMessageDecoder}} extend from
> {{LengthFieldBasedFrameDecoder}} and handle the decoding of the frames and
> the objects in there. In the frame creation otherwise done by
> {{LengthFieldBasedFrameDecoder}}, we could use a sliced buffer instead. This
> solution also makes the channel pipelines a bit simpler.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
