[
https://issues.apache.org/jira/browse/HDFS-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13929749#comment-13929749
]
Brandon Li commented on HDFS-6080:
----------------------------------
[~ashahab], thanks for the patch. Very nice work!
The patch looks good. Some minor comments:
1. According to the discussion in HDFS-6050, we should change
dfs.nfs3.rtmax/dfs.nfs3.wtmax/dfs.nfs3/dtmax to hadoop.nfs.xxx. We can keep
them in NFS3Constants.java for now. HDFS-6056 will move the NFS configuration
property to a different file.
2. HdfsNfsGateway.apt.vm: the property dfs.namenode.accesstime.precision is
pushed down to a wrong place. It might be better to order them like the
following:
{noformat}
* If the client mounts the export with access time update allowed, make sure
the following
... ... .
----
<property>
<name>dfs.namenode.accesstime.precision</name>
<value>3600000</value>
<description>The access time for HDFS file is precise upto this value.
The default value is 1 hour. Setting a value of 0 disables
access times for HDFS.
</description>
</property>
----
* For optimal performance, it is recommended that rtmax be updated to
1MB. However, note that this 1MB is a per client allocation, and not
from a shared memory pool, and therefore a larger value may adversely
affect small reads, consuming a lot of memory. The NFS gateway heap size
can be increased by update HADOOP_NFS3_OPTS in hadoop-env.sh.
The maximum value of this property is 1MB.
----
<property>
<name>hadoop.nfs.rtmax</name>
<value>65536</value>
<description>This is the maximum size in bytes of a READ request
supported by the NFS gateway. If you change this, make sure you
also update the nfs mount's rsize(add rsize= # of bytes to the
mount directive).
</description>
</property>
----
----
<property>
<name>hadoop.nfs.wtmax</name>
<value>65536</value>
<description>This is the maximum size in bytes of a WRITE request
supported by the NFS gateway. If you change this, make sure you
also update the nfs mount's wsize(add wsize= # of bytes to the
mount directive).
</description>
</property>
{noformat}
3. since you already modified RpcProgramNfs3.java, it would be nice to log an
error at line 555 to indicate the link size is larger than max transfer size.
> rtmax and wtmax for NFS-hdfs-gateway should be configurable
> -----------------------------------------------------------
>
> Key: HDFS-6080
> URL: https://issues.apache.org/jira/browse/HDFS-6080
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: nfs, performance
> Reporter: Abin Shahab
> Assignee: Abin Shahab
> Attachments: HDFS-6080.patch
>
>
> Right now rtmax and wtmax are hardcoded in RpcProgramNFS3. These dictate the
> maximum read and write capacity of the server. Therefore, these affect the
> read and write performance.
> We ran performance tests with 1mb, 100mb, and 1GB files. We noticed
> significant performance decline with the size increase when compared to fuse.
> We realized that the issue was with the hardcoded rtmax size(64k).
> When we increased the rtmax to 1MB, we got a 10x improvement in performance.
> NFS reads:
> +---------------+------------+---------------+---------------+---------------+--------------------+----------------------+
> | File | Size | Run 1 | Run 2 | Run 3
> | Average | Std. Dev. |
> | testFile100Mb | 104857600 | 23.131158137 | 19.24552955 | 19.793332866
> | 20.723340184333335 | 1.7172094782219731 |
> | testFile1Gb | 1073741824 | 219.108776636 | 201.064032255 | 217.433909843
> | 212.53557291133333 | 8.14037175506561 |
> | testFile1Mb | 1048576 | 0.330546906 | 0.256391808 | 0.28730168
> | 0.2914134646666667 | 0.030412987573361663 |
> +---------------+------------+---------------+---------------+---------------+--------------------+----------------------+
> Fuse reads:
> +---------------+------------+-------------+--------------+--------------+--------------------+-----------------------+
> | File | Size | Run 1 | Run 2 | Run 3 |
> Average | Std. Dev. |
> | testFile100Mb | 104857600 | 2.394459443 | 2.695265191 | 2.50046517 |
> 2.5300632679999997 | 0.12457410127142007 |
> | testFile1Gb | 1073741824 | 25.03324924 | 24.155102554 | 24.901525525 |
> 24.696625772999997 | 0.386672412437576 |
> | testFile1Mb | 1048576 | 0.271615094 | 0.270835986 | 0.271796438 |
> 0.2714158393333333 | 0.0004166483951065848 |
> +---------------+------------+-------------+--------------+--------------+--------------------+-----------------------+
> (NFS read after rtmax = 1MB)
> +---------------+------------+--------------+-------------+--------------+---------------------+-----------------+
> | File | Size | Run 1 | Run 2 | Run 3 |
> Average | Std. Dev. |
> | testFile100Mb | 104857600 | 3.655261869 | 3.438676067 | 3.557464787 |
> 3.5504675743333336 | 0.0885591069882058 |
> | testFile1Gb | 1073741824 | 34.663612417 | 37.32089122 | 37.997718857 |
> 36.660740831333335 | 1.4389615098060426 |
> | testFile1Mb | 1048576 | 0.115602858 | 0.106826253 | 0.125229976 |
> 0.11588636233333334 | 0.007515962395481867 |
> +---------------+------------+--------------+-------------+--------------+---------------------+-----------------+
--
This message was sent by Atlassian JIRA
(v6.2#6252)
