Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18849#discussion_r133236830
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -1175,6 +1205,27 @@ private[spark] class HiveExternalCatalog(conf:
SparkConf, hadoopConf: Configurat
client.listFunctions(db, pattern)
}
+ /** Detect whether a table is stored with Hive-compatible metadata. */
+ private def isHiveCompatible(table: CatalogTable): Boolean = {
--- End diff --
There's already code to check compatibility in `createDataSourceTable`. The
problem is that the logic there can only really be applied in the context of
that method. Later on you cannot detect compatibility the same way, and just
checking the schema is *not* enough.
The previous code was prone to leave corrupted tables (in old versions of
Hive) or just plain not work (in newer versions of Hive). Having a consistent
way of checking for Hive compatibility is key to that fix, because
`alterTableSchema` has to behave differently for each case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
