[
https://issues.apache.org/jira/browse/SPARK-26835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Horvath updated SPARK-26835:
----------------------------------
Description:
Currently the [Generic Load/Save Functions section of Spark SQL
documentation|https://spark.apache.org/docs/2.4.0/sql-data-sources-load-save-functions.html]
does not explain the available configuration properties at all.
Neither the available formats nor their configuration properties are listed
properly: there are some usage samples, but that is really.
For some formats, there is a remark to visit the site of the provider; quote:
"_To find more detailed information about the extra ORC/Parquet options, visit
the official Apache ORC/Parquet websites."_
However, this is not applicable for all format providers; for example there is
not even a hint regarding the CSV writer's configuration properties or where
they can be looked up.
Please add documentation regarding the configuration properties. Either copy
over documentation completely, or link.
was:
Currently the [Generic Load/Save Functions section of Spark SQL
documentation|https://spark.apache.org/docs/2.4.0/sql-data-sources-load-save-functions.html]
does not explain the available configuration properties at all.
Neither the available formats nor their configuration properties are listed
properly: there are some usage samples, but that is really.
For some formats, there is a remark to visit the site of the provider; quote:
"_To find more detailed information about the extra ORC/Parquet options, visit
the official Apache ORC/Parquet websites."_
However, this is not applicable for all format providers; for example there is
not even a hint regarding the CSV writer's configuration properties or where
they can be looked up.
Please add documentation regarding the configuration properties. Either copy
over documentation completely, or link them.
> Document configuration properties of Spark SQL Generic Load/Save Functions
> --------------------------------------------------------------------------
>
> Key: SPARK-26835
> URL: https://issues.apache.org/jira/browse/SPARK-26835
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Peter Horvath
> Priority: Major
>
> Currently the [Generic Load/Save Functions section of Spark SQL
> documentation|https://spark.apache.org/docs/2.4.0/sql-data-sources-load-save-functions.html]
>
> does not explain the available configuration properties at all.
> Neither the available formats nor their configuration properties are listed
> properly: there are some usage samples, but that is really.
> For some formats, there is a remark to visit the site of the provider; quote:
> "_To find more detailed information about the extra ORC/Parquet options,
> visit the official Apache ORC/Parquet websites."_
> However, this is not applicable for all format providers; for example there
> is not even a hint regarding the CSV writer's configuration properties or
> where they can be looked up.
> Please add documentation regarding the configuration properties. Either copy
> over documentation completely, or link.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
