[
https://issues.apache.org/jira/browse/HDFS-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Li updated HDFS-6080:
-----------------------------
Issue Type: Improvement (was: Bug)
> rtmax and wtmax for NFS-hdfs-gateway should be configurable
> -----------------------------------------------------------
>
> Key: HDFS-6080
> URL: https://issues.apache.org/jira/browse/HDFS-6080
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: nfs
> Reporter: Abin Shahab
> Assignee: Abin Shahab
> Attachments: HDFS-6080.patch
>
>
> Right now rtmax and wtmax are hardcoded in RpcProgramNFS3. These dictate the
> maximum read and write capacity of the server. Therefore, these affect the
> read and write performance.
> We ran performance tests with 1mb, 100mb, and 1GB files. We noticed
> significant performance decline with the size increase when compared to fuse.
> We realized that the issue was with the hardcoded rtmax size(64k).
> When we increased the rtmax to 1MB, we got a 10x improvement in performance.
> NFS reads:
> +---------------+------------+---------------+---------------+---------------+--------------------+----------------------+
> | File | Size | Run 1 | Run 2 | Run 3
> | Average | Std. Dev. |
> | testFile100Mb | 104857600 | 23.131158137 | 19.24552955 | 19.793332866
> | 20.723340184333335 | 1.7172094782219731 |
> | testFile1Gb | 1073741824 | 219.108776636 | 201.064032255 | 217.433909843
> | 212.53557291133333 | 8.14037175506561 |
> | testFile1Mb | 1048576 | 0.330546906 | 0.256391808 | 0.28730168
> | 0.2914134646666667 | 0.030412987573361663 |
> +---------------+------------+---------------+---------------+---------------+--------------------+----------------------+
> Fuse reads:
> +---------------+------------+-------------+--------------+--------------+--------------------+-----------------------+
> | File | Size | Run 1 | Run 2 | Run 3 |
> Average | Std. Dev. |
> | testFile100Mb | 104857600 | 2.394459443 | 2.695265191 | 2.50046517 |
> 2.5300632679999997 | 0.12457410127142007 |
> | testFile1Gb | 1073741824 | 25.03324924 | 24.155102554 | 24.901525525 |
> 24.696625772999997 | 0.386672412437576 |
> | testFile1Mb | 1048576 | 0.271615094 | 0.270835986 | 0.271796438 |
> 0.2714158393333333 | 0.0004166483951065848 |
> +---------------+------------+-------------+--------------+--------------+--------------------+-----------------------+
> (NFS read after rtmax = 1MB)
> +---------------+------------+--------------+-------------+--------------+---------------------+-----------------+
> | File | Size | Run 1 | Run 2 | Run 3 |
> Average | Std. Dev. |
> | testFile100Mb | 104857600 | 3.655261869 | 3.438676067 | 3.557464787 |
> 3.5504675743333336 | 0.0885591069882058 |
> | testFile1Gb | 1073741824 | 34.663612417 | 37.32089122 | 37.997718857 |
> 36.660740831333335 | 1.4389615098060426 |
> | testFile1Mb | 1048576 | 0.115602858 | 0.106826253 | 0.125229976 |
> 0.11588636233333334 | 0.007515962395481867 |
> +---------------+------------+--------------+-------------+--------------+---------------------+-----------------+
--
This message was sent by Atlassian JIRA
(v6.2#6252)
