Repository: spark Updated Branches: refs/heads/master 3ec4461c4 -> 200f01c8f
[SPARK-15760][DOCS] Add documentation for package-related configs. While there, also document spark.files and spark.jars. Text is the same as the spark-submit help text with some minor adjustments. Author: Marcelo Vanzin <[email protected]> Closes #13502 from vanzin/SPARK-15760. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/200f01c8 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/200f01c8 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/200f01c8 Branch: refs/heads/master Commit: 200f01c8fb15680b5630fbd122d44f9b1d096e02 Parents: 3ec4461 Author: Marcelo Vanzin <[email protected]> Authored: Tue Jun 7 09:28:39 2016 -0700 Committer: Marcelo Vanzin <[email protected]> Committed: Tue Jun 7 09:28:39 2016 -0700 ---------------------------------------------------------------------- docs/configuration.md | 47 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/200f01c8/docs/configuration.md ---------------------------------------------------------------------- diff --git a/docs/configuration.md b/docs/configuration.md index d6471a8..32c3a92 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -380,6 +380,53 @@ Apart from these, the following properties are also available, and may be useful from JVM to Python worker for every task. </td> </tr> +<tr> + <td><code>spark.files</code></td> + <td></td> + <td> + Comma-separated list of files to be placed in the working directory of each executor. + </td> +</tr> +<tr> + <td><code>spark.submit.pyFiles</code></td> + <td></td> + <td> + Comma-separated list of .zip, .egg, or .py files to place on the PYTHONPATH for Python apps. + </td> +</tr> +<tr> + <td><code>spark.jars</code></td> + <td></td> + <td> + Comma-separated list of local jars to include on the driver and executor classpaths. + </td> +</tr> +<tr> + <td><code>spark.jars.packages</code></td> + <td></td> + <td> + Comma-separated list of maven coordinates of jars to include on the driver and executor + classpaths. Will search the local maven repo, then maven central and any additional remote + repositories given by <code>spark.jars.ivy</code>. The format for the coordinates should be + groupId:artifactId:version. + </td> +</tr> +<tr> + <td><code>spark.jars.excludes</code></td> + <td></td> + <td> + Comma-separated list of groupId:artifactId, to exclude while resolving the dependencies + provided in <code>spark.jars.packages</code> to avoid dependency conflicts. + </td> +</tr> +<tr> + <td><code>spark.jars.ivy</code></td> + <td></td> + <td> + Comma-separated list of additional remote repositories to search for the coordinates given + with <code>spark.jars.packages</code>. + </td> +</tr> </table> #### Shuffle Behavior --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
