[jira] [Updated] (YARN-9030) Log aggregation changes to handle filesystems which do not support setting permissions

2019-09-04 Thread Adam Antal (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Antal updated YARN-9030:
-
Component/s: log-aggregation

> Log aggregation changes to handle filesystems which do not support setting 
> permissions
> --
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: log-aggregation
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Fix For: 3.3.0, 3.2.1
>
> Attachments: YARN-9030.1.patch, YARN-9030.2.patch
>
>
> Some cloud storages like ADLS do not support permissions in which case they 
> throw an UnsupportedOperationException. Log aggregation code should 
> log/ignore these exceptions and not set permissions henceforth for log 
> aggregation base dir/sub dirs 
> {noformat}
> 2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
> (LogAggregationService.java:initApp(209)) - Application failed to init 
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
> permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
> at 
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-9030) Log aggregation changes to handle filesystems which do not support setting permissions

2018-11-21 Thread Wangda Tan (JIRA) 


 [ 
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-9030:
-
Summary: Log aggregation changes to handle filesystems which do not support 
setting permissions  (was: Log aggregation changes to handle filesystems which 
do not support permissions)

> Log aggregation changes to handle filesystems which do not support setting 
> permissions
> --
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-9030.1.patch, YARN-9030.2.patch
>
>
> Some cloud storages like ADLS do not support permissions in which case they 
> throw an UnsupportedOperationException. Log aggregation code should 
> log/ignore these exceptions and not set permissions henceforth for log 
> aggregation base dir/sub dirs 
> {noformat}
> 2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
> (LogAggregationService.java:initApp(209)) - Application failed to init 
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
> permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
> at 
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org