Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15024#discussion_r85046747
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -741,16 +762,20 @@ object HiveExternalCatalog {
val STATISTICS_NUM_ROWS = STATISTICS_PREFIX + "numRows"
val STATISTICS_COL_STATS_PREFIX = STATISTICS_PREFIX + "colStats."
- def removeStatsProperties(metadata: CatalogTable): Map[String, String] =
{
- metadata.properties.filterNot { case (key, _) =>
key.startsWith(STATISTICS_PREFIX) }
+ // Ideally we should use `spark.sql.sources.location` to store the table
location, but as we have
+ // already used `path` to store it, we should keep it for backward
compatibility.
+ val TABLE_LOCATION = "path"
--- End diff --
the `path` option has special meaning(table location) only when it's used
to create data source tables. Other places like the streaming code path may not
have this semantic.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
