[
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated YARN-9030:
-----------------------------------
Description:
Some cloud storages like ADLS do not support permissions in which case they
throw an UnsupportedOperationException. Log aggregation code should log/ignore
these exceptions and not set permissions henceforth for log aggregation base
dir/sub dirs
{noformat}
2018-11-12 15:37:28,726 WARN logaggregation.LogAggregationService
(LogAggregationService.java:initApp(209)) - Application failed to init
aggregation
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check
permissions for dir [abfs://[email protected]/app-logs]
at
org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)
{noformat}
was:
Some cloud storages like ADLS do not support permissions in which case they
throw an UnsupportedOperationException. Log aggregation should hanlde these
case and not set permissions for log aggregation base dir/ sub dirs
{noformat}
2018-11-12 15:37:28,726 WARN logaggregation.LogAggregationService
(LogAggregationService.java:initApp(209)) - Application failed to init
aggregation
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check
permissions for dir [abfs://[email protected]/app-logs]
at
org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)
{noformat}
> Log aggregation changes to handle filesystems which do not support permissions
> ------------------------------------------------------------------------------
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
> Attachments: YARN-9030.1.patch, YARN-9030.2.patch
>
>
> Some cloud storages like ADLS do not support permissions in which case they
> throw an UnsupportedOperationException. Log aggregation code should
> log/ignore these exceptions and not set permissions henceforth for log
> aggregation base dir/sub dirs
> {noformat}
> 2018-11-12 15:37:28,726 WARN logaggregation.LogAggregationService
> (LogAggregationService.java:initApp(209)) - Application failed to init
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check
> permissions for dir [abfs://[email protected]/app-logs]
> at
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
