[
https://issues.apache.org/jira/browse/RATIS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17294470#comment-17294470
]
Tsz-wo Sze commented on RATIS-1312:
-----------------------------------
Thanks a lot for running the tests!
{quote}2. I did not write disk in streaming and hdfs, to only test performance
of network.
{quote}
How about the performance if it writes to disk?
{quote}1. decodeDataStreamRequestByteBuf costs 6.42% caused by, netty splits
4MB to many small packets, each about 60KB, ...
{quote}
This is probably for TCP packet size.
{quote}2. allocating DirectByteBuffer costs 8.79%, also caused by too many
small packets, ...
{quote}
I am a little surprise that Netty does not manage the DirectByteBuffer(s) well.
There may be some Netty configurations that we should tune.
{quote}If we define buffer size to 60KB, it's slower than 4MB
{quote}
How about if the buffer size is 1MB?
BTW, could you try different numbers of files, say, 1,600, 6,400? (We may need
to decrease the size. Otherwise, the test may take a long time.) HDFS creates
a thread per file. I guess the performance may degrade as the number of files
increases.
> Compare the performance between HDFS and DataStreamApi
> ------------------------------------------------------
>
> Key: RATIS-1312
> URL: https://issues.apache.org/jira/browse/RATIS-1312
> Project: Ratis
> Issue Type: Sub-task
> Reporter: runzhiwang
> Priority: Major
> Attachments: hdfs.svg, streaming.svg
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
