tomicooler commented on code in PR #6120:
URL: https://github.com/apache/hadoop/pull/6120#discussion_r1339564280
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/LogAggregationFileController.java:
##########
@@ -429,26 +460,34 @@ public void verifyAndCreateRemoteLogDir() {
+ remoteRootLogDir + "]", e);
}
} else {
- //Check if FS has capability to set/modify permissions
- Path permissionCheckFile = new Path(qualified,
String.format("%s.permission_check",
- RandomStringUtils.randomAlphanumeric(8)));
+ final FsLogPathKey key = new FsLogPathKey(remoteFS.getClass(),
qualified);
+ FileSystem finalRemoteFS = remoteFS;
+ fsSupportsChmod = FS_CHMOD_CACHE.computeIfAbsent(key,
+ k -> checkFsSupportsChmod(finalRemoteFS, remoteRootLogDir,
qualified));
+ }
+ }
+
+ private boolean checkFsSupportsChmod(FileSystem remoteFS, Path logDir, Path
qualified) {
+ //Check if FS has capability to set/modify permissions
+ Path permissionCheckFile = new Path(qualified,
String.format("%s.permission_check",
+ RandomStringUtils.randomAlphanumeric(8)));
+ try {
+ remoteFS.createNewFile(permissionCheckFile);
+ remoteFS.setPermission(permissionCheckFile, new
FsPermission(TLDIR_PERMISSIONS));
+ return true;
+ } catch (UnsupportedOperationException use) {
+ LOG.info("Unable to set permissions for configured filesystem since"
+ + " it does not support this {}", remoteFS.getScheme());
+ } catch (IOException e) {
+ LOG.warn("Failed to check if FileSystem supports permissions on "
+ + "remoteLogDir [" + logDir + "]", e);
+ } finally {
try {
- remoteFS.createNewFile(permissionCheckFile);
- remoteFS.setPermission(permissionCheckFile, new
FsPermission(TLDIR_PERMISSIONS));
- } catch (UnsupportedOperationException use) {
- LOG.info("Unable to set permissions for configured filesystem since"
- + " it does not support this {}", remoteFS.getScheme());
- fsSupportsChmod = false;
Review Comment:
Note for reviewers. Prior to this change, `fsSupportsChmod` was only set to
false when the `UnsupportedOperationException` was thrown, otherwise
`fsSupportsChmod` was not modified (by default it was set to true).
Now `fsSupportsChmod` can be updated (set to true when everything succeeds)
or false when anything fails. Alternatively we could set it to false only when
`UnsupportedOperationException` is thrown and set it to true only when
everything succeeds otherwise keeping it as-is.
Or keep the original behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
