[
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692967#comment-16692967
]
Suma Shivaprasad commented on YARN-9030:
----------------------------------------
Thanks for the review [~leftnoteasy]. Updated the patch to address review
comments for 4).
To clarify on above comments which are not addressed,
1) Does not throw IOException. Hence need not be inside try block
2) this creates only top level /app-logs directory which should not be created
if it exists - mkdirs should not happen again.
Have tested the patch with both ABFS + HDFS.
> Log aggregation changes to handle filesystems which do not support permissions
> ------------------------------------------------------------------------------
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
> Attachments: YARN-9030.1.patch
>
>
> Some cloud storages like ADLS do not support permissions in which case they
> throw an UnsupportedOperationException. Log aggregation should hanlde these
> case and not set permissions for log aggregation base dir/ sub dirs
> {noformat}
> 2018-11-12 15:37:28,726 WARN logaggregation.LogAggregationService
> (LogAggregationService.java:initApp(209)) - Application failed to init
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check
> permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
> at
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
