dh20 commented on a change in pull request #31030:
URL: https://github.com/apache/spark/pull/31030#discussion_r552342429
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
##########
@@ -130,7 +130,10 @@ private[spark] object HiveUtils extends Logging {
val CONVERT_METASTORE_ORC = buildConf("spark.sql.hive.convertMetastoreOrc")
.doc("When set to true, the built-in ORC reader and writer are used to
process " +
- "ORC tables created by using the HiveQL syntax, instead of Hive serde.")
+ "ORC tables created by using the HiveQL syntax, instead of Hive serde." +
+ "Enabling this parameter will cause insert overwrite when reading the
table," +
+ "and the file format of this table is ORC This error will be thrown[], "
+
Review comment:
@dongjoon-hyun Hello, I'll answer your questions one by one.
- 1.When the format of the table is parquet, there is no problem to insert
and overlay the same table, no matter whether this parameter is added or not【
spark.sql.hive .convertMetastoreParquet=false】
- 2.[PartitionOverwriteMode.DYNAMIC] It can't solve the problem
- 3.I use spark 3.0.0, spark-30112 does not solve this problem.
.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
