This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new fd825d8dbea4 [MINOR][DOCS] Update `CTAS` with `LOCATION` behavior with
Spark 3.2+
fd825d8dbea4 is described below
commit fd825d8dbea4088e8b93271816aaca529ccdb4b2
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sun Oct 8 04:44:55 2023 -0700
[MINOR][DOCS] Update `CTAS` with `LOCATION` behavior with Spark 3.2+
### What changes were proposed in this pull request?
This PR aims to update `CTAS` with `LOCATION` behavior according to Spark
3.2+.
### Why are the changes needed?
SPARK-28551 changed the behavior at Apache Spark 3.2.0.
https://github.com/apache/spark/blob/24b82dfd6cfb9a658af615446be5423695830dd9/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L2306-L2313
### Does this PR introduce _any_ user-facing change?
No. This is a documentation fix.
### How was this patch tested?
N/A
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #43277 from dongjoon-hyun/minor_ctas.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 2d6d09b71e77b362a4c774170e2ca992a31fb1ea)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
docs/sql-ref-syntax-ddl-create-table-datasource.md | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/docs/sql-ref-syntax-ddl-create-table-datasource.md
b/docs/sql-ref-syntax-ddl-create-table-datasource.md
index 7920a8a558e3..f645732a15df 100644
--- a/docs/sql-ref-syntax-ddl-create-table-datasource.md
+++ b/docs/sql-ref-syntax-ddl-create-table-datasource.md
@@ -104,7 +104,9 @@ In general CREATE TABLE is creating a "pointer", and you
need to make sure it po
existing. An exception is file source such as parquet, json. If you don't
specify the LOCATION,
Spark will create a default table location for you.
-For CREATE TABLE AS SELECT, Spark will overwrite the underlying data source
with the data of the
+For CREATE TABLE AS SELECT with LOCATION, Spark throws analysis exceptions if
the given location
+exists as a non-empty directory. If
`spark.sql.legacy.allowNonEmptyLocationInCTAS` is set to true,
+Spark overwrites the underlying data source with the data of the
input query, to make sure the table gets created contains exactly the same
data as the input query.
### Examples
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
