Repository: spark Updated Branches: refs/heads/branch-2.2 ab87a92a1 -> c311c5e79
[MINOR][DOC] automatic type inference supports also Date and Timestamp ## What changes were proposed in this pull request? Easy fix in the documentation, which is reporting that only numeric types and string are supported in type inference for partition columns, while Date and Timestamp are supported too since 2.1.0, thanks to SPARK-17388. ## How was this patch tested? n/a Author: Marco Gaido <mga...@hortonworks.com> Closes #19628 from mgaido91/SPARK-22398. (cherry picked from commit b04eefae49b96e2ef5a8d75334db29ef4e19ce58) Signed-off-by: hyukjinkwon <gurwls...@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c311c5e7 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c311c5e7 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c311c5e7 Branch: refs/heads/branch-2.2 Commit: c311c5e7976a60a0f67d913ca10fe72a8559148f Parents: ab87a92 Author: Marco Gaido <mga...@hortonworks.com> Authored: Thu Nov 2 09:30:03 2017 +0900 Committer: hyukjinkwon <gurwls...@gmail.com> Committed: Thu Nov 2 09:30:20 2017 +0900 ---------------------------------------------------------------------- docs/sql-programming-guide.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/c311c5e7/docs/sql-programming-guide.md ---------------------------------------------------------------------- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 9a54adc..8cd4d05 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -778,10 +778,11 @@ root {% endhighlight %} Notice that the data types of the partitioning columns are automatically inferred. Currently, -numeric data types and string type are supported. Sometimes users may not want to automatically -infer the data types of the partitioning columns. For these use cases, the automatic type inference -can be configured by `spark.sql.sources.partitionColumnTypeInference.enabled`, which is default to -`true`. When type inference is disabled, string type will be used for the partitioning columns. +numeric data types, date, timestamp and string type are supported. Sometimes users may not want +to automatically infer the data types of the partitioning columns. For these use cases, the +automatic type inference can be configured by +`spark.sql.sources.partitionColumnTypeInference.enabled`, which is default to `true`. When type +inference is disabled, string type will be used for the partitioning columns. Starting from Spark 1.6.0, partition discovery only finds partitions under the given paths by default. For the above example, if users pass `path/to/table/gender=male` to either --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org