[
https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15251558#comment-15251558
]
SuYan edited comment on SPARK-14750 at 4/21/16 8:37 AM:
--------------------------------------------------------
historyServer for spark on yarn, the logUrl was something like:
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096
user can refer spark.log for running Application through ExecutorsPage, but
after application was finished, stdout,stderr,gc.log will moved from
nodeManager folder to yarn hdfs folder.
So user can't use
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096
to get application logs, because it was removed.
and I not familiar with spark standalone mode, will user can refer
stdout/stderr/gclog after application was finished?
it can be more convenient if user can refer application log in historyServer,
right?
was (Author: suyan):
historyServer for spark on yarn, the logUrl was something like:
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096
user can refer spark.log for running Application through ExecutorsPage, but
after application was finished, stdout,stderr,gc.log will moved from
nodeManager folder to yarn hdfs folder.
So user can't use
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096
to get application logs, because it was removed.
> Make historyServer refer application log in hdfs
> ------------------------------------------------
>
> Key: SPARK-14750
> URL: https://issues.apache.org/jira/browse/SPARK-14750
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 1.6.1
> Reporter: SuYan
>
> Make history server refer application log, just like MR history server
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
