kihwal commented on a change in pull request #1758: HDFS-15052. WebHDFS
getTrashRoot leads to OOM due to FileSystem objec…
URL: https://github.com/apache/hadoop/pull/1758#discussion_r381340045
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
##########
@@ -1345,11 +1348,21 @@ protected Response get(
}
}
- private static String getTrashRoot(String fullPath,
- Configuration conf) throws IOException {
- FileSystem fs = FileSystem.get(conf != null ? conf : new Configuration());
- return fs.getTrashRoot(
- new org.apache.hadoop.fs.Path(fullPath)).toUri().getPath();
+ private String getTrashRoot(String fullPath) throws IOException {
+ String user = UserGroupInformation.getCurrentUser().getShortUserName();
+ org.apache.hadoop.fs.Path path = new org.apache.hadoop.fs.Path(fullPath);
+ String parentSrc = path.isRoot() ?
+ path.toUri().getPath() : path.getParent().toUri().getPath();
+ EncryptionZone ez = getRpcClientProtocol().getEZForPath(parentSrc);
+ org.apache.hadoop.fs.Path trashRoot;
+ if (ez != null) {
+ trashRoot = new org.apache.hadoop.fs.Path(
+ new org.apache.hadoop.fs.Path(ez.getPath(), TRASH_PREFIX), user);
+ } else {
+ trashRoot = new org.apache.hadoop.fs.Path(
+ new org.apache.hadoop.fs.Path(USER_HOME_PREFIX, user), TRASH_PREFIX);
+ }
+ return trashRoot.toUri().getPath();
Review comment:
Discussed with @daryn-sharp on this. New features in the future might
require additional logic in determining trash root. It is better to be done at
the server-side. So, the Ideal solution would be to add a `getTrashRoot()`
namenode RPC method. `DistributedFileSystem` would then have a fallback logic
for compatibility with older servers. `NamenodeWebHdfsMethods` would simply
call this rpc method.
Also, use of `Path` on NN is highly discouraged. Path is super-expensive and
performs normalization that no other operation performs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
