ASF GitHub Bot commented on FLINK-7428:

Github user NicoK commented on the issue:

    With the recent update of the netty version in flink-shaded (see 
 most of this PR is indirectly included. However, I probably need to fall back 
to the original code creating a copy at least for the fallback non-credit based 
paths due to the original problem.
    Additionally, since the introduction of the credit-based flow control we 
should not use too much memory on the receiver side anymore: memory size in the 
range of once in our buffers, at most one (or two depending on the above) more 
copies in netty temporarily.
    Ultimately, we could go one step further of decoding directly to our 
buffers here, but that is not covered by this PR and will only work with 
credit-based flow control.

> avoid one additional buffer copy when receiving messages
> --------------------------------------------------------
>                 Key: FLINK-7428
>                 URL: https://issues.apache.org/jira/browse/FLINK-7428
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>    Affects Versions: 1.4.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Major
> By using {{LengthFieldBasedFrameDecoder}}, we create one unnecessary (netty) 
> buffer copy in this class which could be easily avoided since we can ensure 
> that the buffer is free to be released after decoding it in the 
> {{NettyMessageDecoder}} and into our own buffer and/or events.
> The solution would be to make {{NettyMessageDecoder}} extend from 
> {{LengthFieldBasedFrameDecoder}} and handle the decoding of the frames and 
> the objects in there. In the frame creation otherwise done by 
> {{LengthFieldBasedFrameDecoder}}, we could use a sliced buffer instead. This 
> solution also makes the channel pipelines a bit simpler.

This message was sent by Atlassian JIRA

Reply via email to