[GitHub] spark pull request #20260: [SPARK-23039][SQL] Finish TODO work in alter tabl...
Github user xubo245 closed the pull request at: https://github.com/apache/spark/pull/20260 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20260: [SPARK-23039][SQL] Finish TODO work in alter tabl...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20260#discussion_r180359425
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -787,7 +787,7 @@ private[spark] class HiveExternalCatalog(conf:
SparkConf, hadoopConf: Configurat
val storageWithLocation = {
val tableLocation = getLocationFromStorageProps(table)
// We pass None as `newPath` here, to remove the path option in
storage properties.
- updateLocationInStorageProps(table, newPath = None).copy(
+ table.storage.copy(
--- End diff --
So we should remove the TODO comment/work?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20260: [SPARK-23039][SQL] Finish TODO work in alter tabl...
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/20260#discussion_r180181331
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -787,7 +787,7 @@ private[spark] class HiveExternalCatalog(conf:
SparkConf, hadoopConf: Configurat
val storageWithLocation = {
val tableLocation = getLocationFromStorageProps(table)
// We pass None as `newPath` here, to remove the path option in
storage properties.
- updateLocationInStorageProps(table, newPath = None).copy(
+ table.storage.copy(
--- End diff --
In the comments above:
```
// Internally we store the table location in storage properties with
key "path" for data
// source tables. Here we set the table location to `locationUri` field
and filter out the
// path option in storage properties, to avoid exposing this concept
externally.
```
```
// We pass None as `newPath` here, to remove the path option in storage
properties.
```
The property `path` is removed intentionally.
To finish to a `TODO` comment in unit test is not a good reason to keep it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20260: [SPARK-23039][SQL] Finish TODO work in alter tabl...
GitHub user xubo245 opened a pull request:
https://github.com/apache/spark/pull/20260
[SPARK-23039][SQL] Finish TODO work in alter table set location.
## What changes were proposed in this pull request?
Finish TODO work in alter table set location.
org.apache.spark.sql.execution.command.DDLSuite#testSetLocation
// TODO(gatorsmile): fix the bug in alter table set location.
//if (isUsingHiveMetastore) {
//assert(storageFormat.properties.get("path") === expected)
// }
fix it by remove newPath = None in
org.apache.spark.sql.hive.HiveExternalCatalog#restoreDataSourceTable
## How was this patch tested?
test("SPARK-23039: check path after SET LOCATION")
Wait for https://github.com/apache/spark/pull/20249
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xubo245/spark setLocationTODO
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20260.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20260
commit 76c1813cf6e0e0e0d085cd31dcf1633c80829eff
Author: xubo245 <601450868@...>
Date: 2018-01-13T13:53:52Z
[SPARK-23039][SQL] Fix the bug in alter table set location.
TOBO work: Fix the bug in alter table set location.
org.apache.spark.sql.execution.command.DDLSuite#testSetLocation
// TODO(gatorsmile): fix the bug in alter table set location.
//if (isUsingHiveMetastore) {
//assert(storageFormat.properties.get("path") === expected)
// }
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
