This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 4d489c8 [SPARK-30289][FOLLOWUP][DOC] Update the migration guide for
`spark.sql.legacy.ctePrecedencePolicy`
4d489c8 is described below
commit 4d489c85aa56816992d12908298e13bfd6fb4a0d
Author: Yuanjian Li <[email protected]>
AuthorDate: Wed Mar 4 13:56:02 2020 +0900
[SPARK-30289][FOLLOWUP][DOC] Update the migration guide for
`spark.sql.legacy.ctePrecedencePolicy`
### What changes were proposed in this pull request?
Fix the migration guide document for
`spark.sql.legacy.ctePrecedence.enabled`, which is introduced in #27579.
### Why are the changes needed?
The config value changed.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Document only.
Closes #27782 from xuanyuanking/SPARK-30829-follow.
Authored-by: Yuanjian Li <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit f7f1948a8c7f7638f34d6a485abc9b866b87700f)
Signed-off-by: HyukjinKwon <[email protected]>
---
docs/sql-migration-guide.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index a9a39ce..0050061 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -101,7 +101,7 @@ license: |
- Since Spark 3.0, if files or subdirectories disappear during recursive
directory listing (i.e. they appear in an intermediate listing but then cannot
be read or listed during later phases of the recursive directory listing, due
to either concurrent file deletions or object store consistency issues) then
the listing will fail with an exception unless
`spark.sql.files.ignoreMissingFiles` is `true` (default `false`). In previous
versions, these missing files or subdirectories would be i [...]
- - Since Spark 3.0, Spark throws an AnalysisException if name conflict is
detected in the nested WITH clause by default. It forces the users to choose
the specific substitution order they wanted, which is controlled by
`spark.sql.legacy.ctePrecedence.enabled`. If set to false (which is
recommended), inner CTE definitions take precedence over outer definitions. For
example, set the config to `false`, `WITH t AS (SELECT 1), t2 AS (WITH t AS
(SELECT 2) SELECT * FROM t) SELECT * FROM t2` re [...]
+ - Since Spark 3.0, `spark.sql.legacy.ctePrecedencePolicy` is introduced to
control the behavior for name conflicting in the nested WITH clause. By default
value `EXCEPTION`, Spark throws an AnalysisException, it forces users to choose
the specific substitution order they wanted. If set to `CORRECTED` (which is
recommended), inner CTE definitions take precedence over outer definitions. For
example, set the config to `false`, `WITH t AS (SELECT 1), t2 AS (WITH t AS
(SELECT 2) SELECT * FR [...]
- Since Spark 3.0, the `add_months` function does not adjust the resulting
date to a last day of month if the original date is a last day of months. For
example, `select add_months(DATE'2019-02-28', 1)` results `2019-03-28`. In
Spark version 2.4 and earlier, the resulting date is adjusted when the original
date is a last day of months. For example, adding a month to `2019-02-28`
results in `2019-03-31`.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
