Github user krisgeus commented on a diff in the pull request:
https://github.com/apache/spark/pull/21893#discussion_r206236120
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -437,6 +437,8 @@ private[hive] class HiveClientImpl(
throw new AnalysisException("Hive index table is not
supported.")
},
schema = schema,
+ hasMultiFormatPartitions =
+ shim.getAllPartitions(client,
h).map(_.getInputFormatClass).distinct.size > 1,
--- End diff --
We could combine this with a configuration setting? This way the user can
enable this at will. working in the same way as
HiveUtils.CONVERT_METASTORE_PARQUET.
What's your thought on that? And do you have a suggestion for config name
;-)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
