10110346 commented on a change in pull request #24174: [SPARK-27238][SQL] Add
fine-grained configurations to handle `convertMetastoreParquet ` and
`convertMetastoreOrc`
URL: https://github.com/apache/spark/pull/24174#discussion_r268472585
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
##########
@@ -97,6 +97,13 @@ private[spark] object HiveUtils extends Logging {
.booleanConf
.createWithDefault(true)
+ val CONVERT_METASTORE_PARQUET_EXCLUDED_TABLES =
+ buildConf("spark.sql.hive.convertMetastoreParquet.excludedTables")
+ .doc("A comma-separated list of Parquet table names, which do not use the
built-in Parquet" +
+ "reader and writer when \"spark.sql.hive.convertMetastoreParquet\" is
true.")
Review comment:
Data is incompatible, but our tests are not in the latest version.
I think that since we have this configuration `spark. sql. hive.
convertMetastoreParquet`, the reason is that it may have incompatible problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
