[
https://issues.apache.org/jira/browse/HDFS-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Li updated HDFS-6365:
-----------------------------
Summary: NFS gateway slow performance when reading big file (~10G) (was:
slow performance when reading big file (~10G))
> NFS gateway slow performance when reading big file (~10G)
> ---------------------------------------------------------
>
> Key: HDFS-6365
> URL: https://issues.apache.org/jira/browse/HDFS-6365
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: nfs
> Affects Versions: 2.4.0
> Reporter: Jifeng Yin
>
> Mount options:
> {code}rw,noatime,nodiratime,vers=3,rsize=65536,wsize=65536,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys{code}
> Read a big file (~10G) ,
> {code}
> time dd if=/file/from/hdfs bs=4M | pv | dd of=/dev/null
> {code}
> Results:
> First time:
> 1. ~160MB/s at first, without cached memory increased on the nfs server
> 2. ~40MB/s with cached memory increased ( cached memory is always kept).
> logs show a ton of the following records:
> {quote}
> 2014-05-09 15:36:13,819 DEBUG org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3:
> READ_RPC_CALL_END______786375175
> 2014-05-09 15:36:13,819 DEBUG org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3:
> READ_RPC_CALL_START____803152391
> 2014-05-09 15:36:13,820 DEBUG org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3:
> NFS READ fileId: 17258 offset: 3243958272 count: 65536
> 2014-05-09 15:36:13,820 DEBUG org.apache.hadoop.hdfs.nfs.nfs3.WriteManager:
> No opened stream for fileId:17258 commitOffset=3244023808. Return success in
> this case.
> {quote}
> Second time:
> the same as first time 2 phrase.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
