[
https://issues.apache.org/jira/browse/RATIS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295731#comment-17295731
]
Tsz-wo Sze commented on RATIS-1312:
-----------------------------------
> ..., ozone mostly write big size file, default 128MB block, ...
One goal of Ozone is to solve the many small file problem. The ability of
supporting more objects (i.e. scalability) is very important in Ozone.
> HDFS can stream write packet to disk and send packet to other datanode at
> size 60KB. Besides, streaming use one socket for all blocks, so each buffer
> size message need to mark which block it belongs to, decode and encode proto
> also cost time. For these two reasons, streaming is slower than HDFS.
The first issue is possible to be fixed it by changing the code (as you
mentioned, consume data ahead, consume one packet when receive one packet).
The second issue is actually an advantage in Streaming since it uses a fewer
sockets, which are a limited system resource. After RATIS-1328, the header
should not affect the performance much.
BTW, let's try 1MB buffer.
> Compare the performance between HDFS and DataStreamApi
> ------------------------------------------------------
>
> Key: RATIS-1312
> URL: https://issues.apache.org/jira/browse/RATIS-1312
> Project: Ratis
> Issue Type: Sub-task
> Reporter: runzhiwang
> Priority: Major
> Attachments: hdfs.svg, screenshot-1.png, streaming.svg
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
