HyukjinKwon commented on a change in pull request #24174: [SPARK-27238][SQL]
Add fine-grained configurations to handle `convertMetastoreParquet ` and
`convertMetastoreOrc`
URL: https://github.com/apache/spark/pull/24174#discussion_r268467799
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
##########
@@ -97,6 +97,13 @@ private[spark] object HiveUtils extends Logging {
.booleanConf
.createWithDefault(true)
+ val CONVERT_METASTORE_PARQUET_EXCLUDED_TABLES =
+ buildConf("spark.sql.hive.convertMetastoreParquet.excludedTables")
+ .doc("A comma-separated list of Parquet table names, which do not use the
built-in Parquet" +
+ "reader and writer when \"spark.sql.hive.convertMetastoreParquet\" is
true.")
Review comment:
Why don't we simply disable `spark.sql.hive.convertMetastoreParquet` in this
case? It looks going to make the codes super complicated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
