[
https://issues.apache.org/jira/browse/HDFS-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18046857#comment-18046857
]
ASF GitHub Bot commented on HDFS-14738:
---------------------------------------
github-actions[bot] closed pull request #1300: HDFS-14738：Reading and writing
HDFS data through HDFS nfs3 server often getting stuck
URL: https://github.com/apache/hadoop/pull/1300
> Reading and writing HDFS data through HDFS nfs3 server often getting stuck
> --------------------------------------------------------------------------
>
> Key: HDFS-14738
> URL: https://issues.apache.org/jira/browse/HDFS-14738
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: nfs
> Affects Versions: 2.7.2
> Reporter: liying
> Assignee: liying
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-14738.001.patch
>
>
> We know the NFS Gateway supports NFSv3 and allows HDFS to be mounted as part
> of the client’s local file system.
> I start the portmap and nfs3server in the hadoop node，and use the command
> （mount -t nfs -o vers=3,proto=tcp,nolock nfs3serverIP:/ /hdfs） to mount.
> It works well。Then i ues this command to mount on many client.
> And i used linux cp to copy files to the mounted dir(/hdfs), found the speed
> was very slow . There is a lot of information in the log as the follow:
> 2019-08-14 15:36:28,347 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:03,093 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:03,850 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:14,780 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:14,928 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:28,410 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:09,310 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:10,069 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:14,856 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:14,957 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:28,475 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:14,923 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:14,987 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:15,541 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:16,287 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:28,530 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:15,015 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:15,024 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:15,028 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:21,757 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:22,508 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:28,583 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:15,063 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:15,086 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:27,983 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:28,647 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:28,729 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:42:15,184 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:42:15,191 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:42:28,707 INFO org.apache.hadoop.security.ShellBasedIdMapping:
> Can't map group supergroup. Use its string hashcode:-1710818332
>
> Then i print the jstack log of nfs3 server jvm, and i found that many threads
> waiting for monitor entry : BLOCKED(on object monitor). After checking the
> codes of ShellBasedIdMapping, found that It will invoke the updateMapInternal
> every time if the user or group doesn't exits. And many methods in this class
> are made with synchronized. So i think we can avoid to invoke
> updateMapInternal every time.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
