This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 0fc97b5 [SPARK-34577][SQL][FOLLOWUP] Add change of `DESC NAMESPACE`'s
schema to migration guide
0fc97b5 is described below
commit 0fc97b5bf44e4f9f0faf011a2d12e11af1cdb972
Author: Angerszhuuuu <angers....@gmail.com>
AuthorDate: Wed Apr 14 06:30:45 2021 +0000
[SPARK-34577][SQL][FOLLOWUP] Add change of `DESC NAMESPACE`'s schema to
migration guide
### What changes were proposed in this pull request?
Add change of `DESC NAMESPACE`'s schema to migration guide
### Why are the changes needed?
Update doc
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Not need
Closes #32155 from AngersZhuuuu/SPARK-34577-followup.
Authored-by: Angerszhuuuu <angers....@gmail.com>
Signed-off-by: Wenchen Fan <wenc...@databricks.com>
---
docs/sql-migration-guide.md | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index 34933d2..ffc3814 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -49,6 +49,10 @@ license: |
- In Spark 3.2, the output schema of `SHOW TABLE EXTENDED` becomes
`namespace: string, tableName: string, isTemporary: boolean, information:
string`. In Spark 3.1 or earlier, the `namespace` field was named `database`
for the builtin catalog, and no change for the v2 catalogs. To restore the old
schema with the builtin catalog, you can set
`spark.sql.legacy.keepCommandOutputSchema` to `true`.
+ - In Spark 3.2, the output schema of `SHOW TBLPROPERTIES` becomes `key:
string, value: string` whether you specify the table property key or not. In
Spark 3.1 and earlier, the output schema of `SHOW TBLPROPERTIES` is `value:
string` when you specify the table property key. To restore the old schema with
the builtin catalog, you can set `spark.sql.legacy.keepCommandOutputSchema` to
`true`.
+
+ - In Spark 3.2, the output schema of `DESCRIBE NAMESPACE` becomes
`info_name: string, info_value: string`. In Spark 3.1 or earlier, the
`info_name` field was named `database_description_item` and the `info_value`
field was named `database_description_value` for the builtin catalog. To
restore the old schema with the builtin catalog, you can set
`spark.sql.legacy.keepCommandOutputSchema` to `true`.
+
- In Spark 3.2, table refreshing clears cached data of the table as well as
of all its dependents such as views while keeping the dependents cached. The
following commands perform table refreshing:
* `ALTER TABLE .. ADD PARTITION`
* `ALTER TABLE .. RENAME PARTITION`
@@ -62,8 +66,6 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid
producing ambiguous results. Because `count(*)` and `count(tblName.*)` will
output differently if there is any null values. To restore the behavior before
Spark 3.2, you can set
`spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
-
- - In Spark 3.2, the output schema of `SHOW TBLPROPERTIES` becomes `key:
string, value: string` whether you specify the table property key or not. In
Spark 3.1 and earlier, the output schema of `SHOW TBLPROPERTIES` is `value:
string` when you specify the table property key. To restore the old schema with
the builtin catalog, you can set `spark.sql.legacy.keepCommandOutputSchema` to
`true`.
- In Spark 3.2, we support typed literals in the partition spec of INSERT
and ADD/DROP/RENAME PARTITION. For example, `ADD PARTITION(dt =
date'2020-01-01')` adds a partition with date value `2020-01-01`. In Spark 3.1
and earlier, the partition value will be parsed as string value `date
'2020-01-01'`, which is an illegal date value, and we add a partition with null
value at the end.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
