[
https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peng Zhang updated SPARK-2668:
------------------------------
Summary: Add variable of yarn log directory for reference from the log4j
configuration (was: Add variable of yarn log directory to reference from the
log4j configuration)
> Add variable of yarn log directory for reference from the log4j configuration
> -----------------------------------------------------------------------------
>
> Key: SPARK-2668
> URL: https://issues.apache.org/jira/browse/SPARK-2668
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.0.0
> Reporter: Peng Zhang
>
> Assign value of yarn container log directory to java opts
> "spark.yarn.log.dir", So user defined log4j.properties can reference this
> value and write log to YARN container directory.
> Otherwise, user defined file appender will only write to container's CWD, and
> log files in CWD will not be displayed on YARN UIļ¼and either cannot be
> aggregated to HDFS log directory after job finished.
> User defined log4j.properties reference example:
> {code}
> log4j.appender.rolling_file.File = ${spark.yarn.log.dir}/spark.log
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
