[
https://issues.apache.org/jira/browse/LIVY-870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643120#comment-17643120
]
Akshat Bordia commented on LIVY-870:
------------------------------------
Created PR for review: [[LIVY-870] Adding changes to provide Spark config using
--properties-file by akshatb1 · Pull Request #331 · apache/incubator-livy ·
GitHub|https://github.com/apache/incubator-livy/pull/331].
> Provide Spark config via --properties-file for Batch Jobs
> ---------------------------------------------------------
>
> Key: LIVY-870
> URL: https://issues.apache.org/jira/browse/LIVY-870
> Project: Livy
> Issue Type: Improvement
> Reporter: Akshat Bordia
> Priority: Major
> Fix For: 0.9.0
>
>
> Currently for batch jobs, individual Spark configs are added as --conf
> parameter in Spark submit command line. If there are many Spark
> configurations added in Livy APIs, it might increase the length of command
> line and in some cases might cause issues due to limit on command line length
> (Windows has 8191 character length limit on the command line length.).
>
> We can merge the configurations into a temporary properties file and add that
> as --properties-file opt in spark-submit command
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
