[
https://issues.apache.org/jira/browse/SPARK-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558540#comment-15558540
]
holdenk commented on SPARK-14212:
---------------------------------
So I think this would be a good option to document for Python users, although
the root CSV issue has been fixed by including the CSV format inside of Spark
its self. You configure a package using `spark.jars.packages` in
spark-defaults.conf.
If someone is interested adding this to the documentation
`docs/configuration.md` would probably be a good place to document the
`spakr.jars.packages` configuration value (and you can see how it can be by
looking at the inside of SparkSubmit & SparkSubmitArguments together.
> Add configuration element for --packages option
> -----------------------------------------------
>
> Key: SPARK-14212
> URL: https://issues.apache.org/jira/browse/SPARK-14212
> Project: Spark
> Issue Type: New Feature
> Components: Documentation, PySpark
> Affects Versions: 1.6.1
> Reporter: Russell Jurney
> Labels: config, fun, happy, pants, spark-shell
>
> I use PySpark with the --packages option, for instance to load support for
> CSV:
> pyspark --packages com.databricks:spark-csv_2.10:1.4.0
> I would like to not have to set this every time at the command line, so a
> corresponding element for --packages in the configuration file
> spark-defaults.conf, would be good to have.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
