[jira] [Updated] (YARN-9030) Log aggregation changes to handle filesystems which do not support permissions

2018-11-20 Thread Suma Shivaprasad (JIRA) 


 [ 
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated YARN-9030:
---
Description: 
Some cloud storages like ADLS do not support permissions in which case they 
throw an UnsupportedOperationException. Log aggregation code should log/ignore 
these exceptions and not set permissions henceforth for log aggregation base 
dir/sub dirs 

{noformat}

2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
(LogAggregationService.java:initApp(209)) - Application failed to init 
aggregation
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
at 
org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)

{noformat}

  was:
Some cloud storages like ADLS do not support permissions in which case they 
throw an UnsupportedOperationException. Log aggregation should hanlde these 
case and not set permissions for log aggregation base dir/ sub dirs 

{noformat}

2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
(LogAggregationService.java:initApp(209)) - Application failed to init 
aggregation
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
at 
org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)

{noformat}


> Log aggregation changes to handle filesystems which do not support permissions
> --
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-9030.1.patch, YARN-9030.2.patch
>
>
> Some cloud storages like ADLS do not support permissions in which case they 
> throw an UnsupportedOperationException. Log aggregation code should 
> log/ignore these exceptions and not set permissions henceforth for log 
> aggregation base dir/sub dirs 
> {noformat}
> 2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
> (LogAggregationService.java:initApp(209)) - Application failed to init 
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
> permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
> at 
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at 
>

[jira] [Updated] (YARN-9030) Log aggregation changes to handle filesystems which do not support permissions

2018-11-20 Thread Suma Shivaprasad (JIRA) 


 [ 
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated YARN-9030:
---
Attachment: YARN-9030.2.patch

> Log aggregation changes to handle filesystems which do not support permissions
> --
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-9030.1.patch, YARN-9030.2.patch
>
>
> Some cloud storages like ADLS do not support permissions in which case they 
> throw an UnsupportedOperationException. Log aggregation should hanlde these 
> case and not set permissions for log aggregation base dir/ sub dirs 
> {noformat}
> 2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
> (LogAggregationService.java:initApp(209)) - Application failed to init 
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
> permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
> at 
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-9030) Log aggregation changes to handle filesystems which do not support permissions

2018-11-16 Thread Suma Shivaprasad (JIRA) 


 [ 
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated YARN-9030:
---
Attachment: YARN-9030.1.patch

> Log aggregation changes to handle filesystems which do not support permissions
> --
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-9030.1.patch
>
>
> Some cloud storages like ADLS do not support permissions in which case they 
> throw an UnsupportedOperationException. Log aggregation should hanlde these 
> case and not set permissions for log aggregation base dir/ sub dirs 
> {noformat}
> 2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
> (LogAggregationService.java:initApp(209)) - Application failed to init 
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
> permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
> at 
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-9030) Log aggregation changes to handle filesystems which do not support permissions

2018-11-16 Thread Suma Shivaprasad (JIRA) 


 [ 
https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated YARN-9030:
---
Description: 
Some cloud storages like ADLS do not support permissions in which case they 
throw an UnsupportedOperationException. Log aggregation should hanlde these 
case and not set permissions for log aggregation base dir/ sub dirs 

{noformat}

2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
(LogAggregationService.java:initApp(209)) - Application failed to init 
aggregation
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
at 
org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)

{noformat}

  was:
Some cloud storages like ABFS do not support permissions in which case they 
throw an UnsupportedOperationException. Log aggregation should hanlde these 
case and not set permissions for log aggregation base dir/ sub dirs 

{noformat}

2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
(LogAggregationService.java:initApp(209)) - Application failed to init 
aggregation
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
at 
org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at 
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)

{noformat}


> Log aggregation changes to handle filesystems which do not support permissions
> --
>
> Key: YARN-9030
> URL: https://issues.apache.org/jira/browse/YARN-9030
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
>
> Some cloud storages like ADLS do not support permissions in which case they 
> throw an UnsupportedOperationException. Log aggregation should hanlde these 
> case and not set permissions for log aggregation base dir/ sub dirs 
> {noformat}
> 2018-11-12 15:37:28,726 WARN  logaggregation.LogAggregationService 
> (LogAggregationService.java:initApp(209)) - Application failed to init 
> aggregation
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check 
> permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
> at 
> org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
> at 
>