[
https://issues.apache.org/jira/browse/RATIS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17294486#comment-17294486
]
runzhiwang edited comment on RATIS-1312 at 3/3/21, 11:42 AM:
-------------------------------------------------------------
> I am a little surprise that Netty does not manage the DirectByteBuffer(s)
> well. There may be some Netty configurations that we should tune.
[~szetszwo] Maybe it's hard for netty to manage the DirectByteBuffer(s)
well. Because when buffer size is 4MB, we only consume data when we receive 4MB
/ 60KB packets, i.e. about 67 packets, it need some time to receive all these
67 packets. Before netty receive all these 67 packets, the max 4MB
directbytebuffer can not be recycled. So if we write 1000 * 4MB at the same
time, then we need a lot of directbytebuffer.
So I'm thinking can we consume data ahead, before we receive all these 67
packets, but maybe hard to do that, because we do not know which the following
packet belongs to.
was (Author: yjxxtd):
> I am a little surprise that Netty does not manage the DirectByteBuffer(s)
> well. There may be some Netty configurations that we should tune.
[~szetszwo] Maybe it's hard for netty to manage the DirectByteBuffer(s)
well. Because when buffer size is 4MB, we only consume data when we receive 4MB
/ 60KB packets, i.e. about 67 packets, it need some time to receive all these
67 packets. Before netty receive all these 67 packets, the max 4MB
directbytebuffer can not be recycled. So if we write 1000 * 4MB at the same
time, then we need a lot of directbytebuffer.
So I'm thinking can we consume data ahead, before we receive all these 67
packets, but maybe hard to do that, because we do not know which the following
packet belongs to.
> Compare the performance between HDFS and DataStreamApi
> ------------------------------------------------------
>
> Key: RATIS-1312
> URL: https://issues.apache.org/jira/browse/RATIS-1312
> Project: Ratis
> Issue Type: Sub-task
> Reporter: runzhiwang
> Priority: Major
> Attachments: hdfs.svg, streaming.svg
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
