[
https://issues.apache.org/jira/browse/SPARK-14602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-14602:
------------------------------------
Assignee: Apache Spark
> [YARN+Windows] Setting SPARK_YARN_CACHE_FILES exceeds command line length
> limit on Windows
> ------------------------------------------------------------------------------------------
>
> Key: SPARK-14602
> URL: https://issues.apache.org/jira/browse/SPARK-14602
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Spark Submit, Windows, YARN
> Affects Versions: 2.0.0
> Environment: YARN on Windows
> Reporter: Sebastian Kochman
> Assignee: Apache Spark
>
> After change https://issues.apache.org/jira/browse/SPARK-11157, which removed
> a single large Spark assembly in favor of multiple small jars, when you try
> to submit a Spark app to YARN on Windows (using spark-submit.cmd), the app
> fails with the following error:
> Diagnostics: The command line has a length of 12046 exceeds maximum allowed
> length of 8191. Command starts with: @set
> SPARK_YARN_CACHE_FILES=[...]/.sparkStaging/application_[...] Failing this
> attempt. Failing the application.
> So basically a large number of jars needed for staging in YARN causes
> exceeding Windows command line length limit.
> Please see more details in the discussion here:
> https://issues.apache.org/jira/browse/SPARK-11157?focusedCommentId=15238151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15238151
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
