[jira] [Commented] (SPARK-2668) Add variable of yarn log diectory to reference from the log4j configuration

[Peng Zhang (JIRA)]

    [ 
https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073982#comment-14073982
 ] 

Peng Zhang commented on SPARK-2668:
-----------------------------------

I changed the title to make it more clear.

And I found MapReduce also added variable 
"-Dyarn.app.mapreduce.container.log.dir" to support reference from log4j 
configuration.
So I think Spark on YARN should have one to solve the same issue.

> Add variable of yarn log diectory to reference from the log4j configuration
> ---------------------------------------------------------------------------
>
>                 Key: SPARK-2668
>                 URL: https://issues.apache.org/jira/browse/SPARK-2668
>             Project: Spark
>          Issue Type: Improvement
>          Components: YARN
>    Affects Versions: 1.0.0
>            Reporter: Peng Zhang
>
> Assign value of yarn container log directory to java opts 
> "spark.yarn.log.dir", So user defined log4j.properties can reference this 
> value and write log to YARN container directory.
> Otherwise, user defined file append will log to CWD, and files will not be 
> displayed on YARN UI，and either cannot be aggregated to HDFS log directory 
> after job finished.
> User defined log4j.properties reference example:
> {code}
> log4j.appender.rolling_file.File = ${spark.yarn.log.dir}/spark.log
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)