[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124529#comment-16124529 ] Mahesh Ambule commented on SPARK-21711: --- Sean Owen: I figured out a way to configure the log4j configuration for spark-client/launcher. The $SPARK_SUBMIT_OPTS environment variable can be set to include log4j configuration. $SPARK_SUBMIT_OPTS gets appended to command line options when "org.apache.spark.deploy.SparkSubmit" is invoked. export SPARK_SUBMIT_OPTS="$SPARK_SUBMIT_OPTS -Dlog4j.configuration=/home/mahesh/log4j.config" Link for the relevant code: https://github.com/apache/spark/blob/branch-2.1/launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java#L242 > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Comment: was deleted (was: Here by spark client, I meant java client process started by spark-submit. I want to provide Java options to this client process. I am talking about java client which invokes yarn client process and launches the driver and executor processes.) > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124457#comment-16124457 ] Mahesh Ambule commented on SPARK-21711: --- My Application > spark-submit > spark client ---> yarn client ---> driver ---> executors I want to configure log4j Java options for spark client and yarn client, but spark-submit does not have the option to configure it. > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123967#comment-16123967 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 8:18 PM: Here by spark client, I meant java client process started by spark-submit. I want to provide Java options to this client process. I am talking about java client which invokes yarn client process and launches the driver and executor processes. was (Author: mahesh_ambule): Here by spark client, I meant java client process started by spark-submit. I want to provide Java options to that process. I am talking about java client which invokes yarn client process and launches the driver and executor processes. > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123967#comment-16123967 ] Mahesh Ambule commented on SPARK-21711: --- Here by spark client, I meant java client process started by spark-submit. I want to provide Java options to that process. I am talking about java client which invokes yarn client process and launches the driver and executor processes. > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123941#comment-16123941 ] Mahesh Ambule commented on SPARK-21711: --- I dont want to configure executor or driver java options. I want to configure spark client java options. > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 7:36 PM: @Sean Owen: Thanks for the reply. I tried to pass '-Dlog4j.configuration=' param to spark-submit command but it did not configure spark client logging. Log4j configuration parameters passed through spark-submit command are getting configured for driver and executor JVMs but not for the spark client JVM. Here I am more interested in spark client application logs. For more clarity, please find attached the sample spark client logs file. I went through spark-submit and related script files and found that spark-submit does not provide the option to pass parameters to client application JVM. Below is the code snippet from '$spark_home/bin/spark-class' file. This 'spark-class' file gets invoked by spark-submit. _ "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"_ Here $RUNNER is 'java' command and '$@' are arguments passed to spark-submit. There is no way to pass parameters to this java JVM. Is my understanding correct? And can the option be provided to pass parameters to client Java JVM? was (Author: mahesh_ambule): Sean Owen: Thanks for the reply. I tried to pass '-Dlog4j.configuration=' param to spark-submit command but it did not configure spark client logging. Log4j configuration parameters passed through spark-submit command are getting configured for driver and executor JVMs but not for the spark client JVM. Here I am more interested in spark client application logs. For more clarity, please find attached the sample spark client logs file. I went through spark-submit and related script files and found that spark-submit does not provide the option to pass parameters to client application JVM. Below is the code snippet from '$spark_home/bin/spark-class' file. This 'spark-class' file gets invoked by spark-submit. _ "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"_ Here $RUNNER is 'java' command and '$@' are arguments passed to spark-submit. There is no way to pass parameters to this java JVM. Is my understanding correct? And can the option be provided to pass parameters to client Java JVM? > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 7:27 PM: Sean Owen: Thanks for the reply. I tried to pass '-Dlog4j.configuration=' param to spark-submit command but it did not configure spark client logging. Log4j configuration parameters passed through spark-submit command are getting configured for driver and executor JVMs but not for the spark client JVM. Here I am more interested in spark client application logs. For more clarity, please find attached the sample spark client logs file. I went through spark-submit and related script files and found that spark-submit does not provide the option to pass parameters to client application JVM. Below is the code snippet from '$spark_home/bin/spark-class' file. This 'spark-class' file gets invoked by spark-submit. _ "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"_ Here $RUNNER is 'java' command and '$@' are arguments passed to spark-submit. There is no way to pass parameters to this java JVM. Is my understanding correct? And can the option be provided to pass parameters to client Java JVM? was (Author: mahesh_ambule): Sean Owen: Thanks for the reply. I tried to pass '-Dlog4j.configuration=' param to spark-submit command but it did not configure spark client logging. Log4j configuration parameters passed through spark-submit command are getting configured for driver and executor JVMs but not for the spark client JVM. Here I am more interested in spark client application logs. For more clarity, please find attached the sample spark client logs file. I went through spark-submit and related script files and found that spark-submit does not provide the option to pass parameters to client application JVM. Below is the code snippet from '$spark_home/bin/spark-class' file. This 'spark-class' file gets invoked by spark-submit. _ "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"_ Here $RUNNER is 'java' command and '$@' are arguments passed to spark-submit. There is no way to pass parameters to client java JVM. Is my understanding correct? And can the option be provided to pass parameters to client Java JVM? > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 7:22 PM: Sean Owen: Thanks for the reply. I tried to pass '-Dlog4j.configuration=' param to spark-submit command but it did not configure spark client logging. Log4j configuration parameters passed through spark-submit command are getting configured for driver and executor JVMs but not for the spark client JVM. Here I am more interested in spark client application logs. For more clarity, please find attached the sample spark client logs file. I went through spark-submit and related script files and found that spark-submit does not provide the option to pass parameters to client application JVM. Below is the code snippet from '$spark_home/bin/spark-class' file. This 'spark-class' file gets invoked by spark-submit. _ "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"_ Here $RUNNER is 'java' command and '$@' are arguments passed to spark-submit. There is no way to pass parameters to client java JVM. Is my understanding correct? And can the option be provided to pass parameters to client Java JVM? was (Author: mahesh_ambule): sean owen: Thanks for the reply. I tried to pass '-Dlog4j.configuration=' param to spark-submit command but it did not configure spark client logging. Log4j configuration parameters passed through spark-submit command are getting configured for driver and executor JVMs but not for the spark client JVM. Here I am more interested in spark client application logs. For more clarity, please find attached the sample spark client logs file. I went through spark-submit and related script files and found that spark-submit does not provide the option to pass parameters to client application JVM. Below is the code snippet from '$spark_home/bin/spark-class' file. This 'spark-class' file gets invoked by spark-submit. _ "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"_ Here $RUNNER is 'java' command and '$@' are arguments passed to spark-submit. There is no way to pass parameters to client java JVM. Is my understanding correct? And can the option be provided to pass parameters to client Java JVM? > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Attachment: spark-submit client logs.txt > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule commented on SPARK-21711: --- sean owen: Thanks for the reply. I tried to pass '-Dlog4j.configuration=' param to spark-submit command but it did not configure spark client logging. Log4j configuration parameters passed through spark-submit command are getting configured for driver and executor JVMs but not for the spark client JVM. Here I am more interested in spark client application logs. For more clarity, please find attached the sample spark client logs file. I went through spark-submit and related script files and found that spark-submit does not provide the option to pass parameters to client application JVM. Below is the code snippet from '$spark_home/bin/spark-class' file. This 'spark-class' file gets invoked by spark-submit. _ "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"_ Here $RUNNER is 'java' command and '$@' are arguments passed to spark-submit. There is no way to pass parameters to client java JVM. Is my understanding correct? And can the option be provided to pass parameters to client Java JVM? > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > Attachments: spark-submit client logs.txt > > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Description: Currently, log4j properties can be specified in spark 'conf' directory in log4j.properties file. The spark-submit command can override these log4j properties for driver and executors. But it can not override these log4j properties for *spark client * application. The user should be able to pass log4j properties for spark client using the spark-submit command. was: Currently, log4j properties can be specified in spark 'conf' directory in log4j.properties file. The spark-submit command can override these log4j properties for driver and executors logs. But it can not override log4j properties for spark and yarn clients in yarn-cluster mode. The user should be able to pass log4j properties for spark and yarn clients using the spark-submit command. > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors. > But it can not override these log4j properties for *spark client * > application. > The user should be able to pass log4j properties for spark client using the > spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Description: Currently, log4j properties can be specified in spark 'conf' directory in log4j.properties file. The spark-submit command can override these log4j properties for driver and executors logs. But it can not override log4j properties for spark and yarn clients in yarn-cluster mode. The user should be able to pass log4j properties for spark and yarn clients using the spark-submit command. was: Currently, log4j properties can be specified in spark 'conf' directory in log4j.properties file. The spark-submit command can override these log4j properties for driver and executors logs. But it can not override log4j properties for spark and yarn clients in yarn-cluster mode. The user should be able to pass log4j properties for spark and yarn client using spark-submit command. > spark-submit command should accept log4j configuration parameters for spark > client logging. > --- > > Key: SPARK-21711 > URL: https://issues.apache.org/jira/browse/SPARK-21711 > Project: Spark > Issue Type: Improvement > Components: Spark Submit >Affects Versions: 1.6.0, 2.1.0 >Reporter: Mahesh Ambule >Priority: Minor > > Currently, log4j properties can be specified in spark 'conf' directory in > log4j.properties file. > The spark-submit command can override these log4j properties for driver and > executors logs. But it can not override log4j properties for spark and yarn > clients in yarn-cluster mode. > The user should be able to pass log4j properties for spark and yarn clients > using the spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.
Mahesh Ambule created SPARK-21711: - Summary: spark-submit command should accept log4j configuration parameters for spark client logging. Key: SPARK-21711 URL: https://issues.apache.org/jira/browse/SPARK-21711 Project: Spark Issue Type: Improvement Components: Spark Submit Affects Versions: 2.1.0, 1.6.0 Reporter: Mahesh Ambule Priority: Minor Currently, log4j properties can be specified in spark 'conf' directory in log4j.properties file. The spark-submit command can override these log4j properties for driver and executors logs. But it can not override log4j properties for spark and yarn clients in yarn-cluster mode. The user should be able to pass log4j properties for spark and yarn client using spark-submit command. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
