[
https://issues.apache.org/jira/browse/HDFS-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Liang updated HDFS-1583:
------------------------------
Attachment: HDFS-1583-1.patch
Attach a patch which wraps Journal function's parameters with a writable class
_JournalArgs_. The following table shows the performance number of RPC call
with and without this patch.
The contents of the form is each rpc call's average time which is total
elapse(ms) divided by 10,000 times.
|array size|1k|2k|3k|4k|5k|6k|7k|8k|
|without patch|1.2212|1.9266|2.5415|3.2025|4.8677|4.5679|5.2211|5.9386|
|with patch|0.4774|0.4087|0.4521|0.4375|0.4215|0.4616|0.4551|0.4844|
> Improve backup-node sync performance by wrapping RPC parameters
> ---------------------------------------------------------------
>
> Key: HDFS-1583
> URL: https://issues.apache.org/jira/browse/HDFS-1583
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Liyin Liang
> Attachments: HDFS-1583-1.patch
>
>
> The journal edit records are sent by the active name-node to the backup-node
> with RPC:
> {code:}
> public void journal(NamenodeRegistration registration,
> int jAction,
> int length,
> byte[] records) throws IOException;
> {code}
> During the name-node throughput benchmark, the size of byte array _records_
> is around *8000*. Then the serialization and deserialization is
> time-consuming. I wrote a simple application to test RPC with byte array
> parameter. When the size got to 8000, each RPC call need about 6 ms. While
> name-node sync 8k byte to local disk only need 0.3~0.4ms.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
