[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099477#comment-16099477 ]
HanCheol Cho commented on SPARK-16784: -------------------------------------- Hi, I used the following options that allows both driver & executors use a custom log4j.properties, {code} spark-submit \ --driver-java-options=-Dlog4j.configuration=conf/log4j.properties \ --files conf/log4j.properties \ --conf "spark.executor.extraJavaOptions='-Dlog4j.configuration=log4j.properties'" \ ... {code} I used a local log4j.properties file for the driver and the file in the distributed cache (by --files option) for the executors. As shown above, the paths to driver and executors are different. I hope it might be useful to the others. > Configurable log4j settings > --------------------------- > > Key: SPARK-16784 > URL: https://issues.apache.org/jira/browse/SPARK-16784 > Project: Spark > Issue Type: Improvement > Affects Versions: 2.0.0, 2.1.0 > Reporter: Michael Gummelt > > I often want to change the logging configuration on a single spark job. This > is easy in client mode. I just modify log4j.properties. It's difficult in > cluster mode, because I need to modify the log4j.properties in the > distribution in which the driver runs. I'd like a way of setting this > dynamically, such as a java system property. Some brief searching showed > that log4j doesn't seem to accept such a property, but I'd like to open up > this idea for further comment. Maybe we can find a solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org