[
https://issues.apache.org/jira/browse/SPARK-8405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-8405.
------------------------------
Resolution: Not A Problem
> Show executor logs on Web UI when Yarn log aggregation is enabled
> -----------------------------------------------------------------
>
> Key: SPARK-8405
> URL: https://issues.apache.org/jira/browse/SPARK-8405
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.4.0
> Reporter: Carson Wang
> Attachments: SparkLogError.png
>
>
> When running Spark application in Yarn mode and Yarn log aggregation is
> enabled, customer is not able to view executor logs on the history server Web
> UI. The only way for customer to view the logs is through the Yarn command
> "yarn logs -applicationId <appId>".
> An screenshot of the error is attached. When you click an executor’s log link
> on the Spark history server, you’ll see the error if Yarn log aggregation is
> enabled. The log URL redirects user to the node manager’s UI. This works if
> the logs are located on that node. But since log aggregation is enabled, the
> local logs are deleted once log aggregation is completed.
> The logs should be available through the web UIs just like other Hadoop
> components like MapReduce. For security reasons, end users may not be able to
> log into the nodes and run the yarn logs -applicationId command. The web UIs
> can be viewable and exposed through the firewall if necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
