dongjoon-hyun commented on a change in pull request #31030:
URL: https://github.com/apache/spark/pull/31030#discussion_r552353495
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
##########
@@ -130,7 +130,10 @@ private[spark] object HiveUtils extends Logging {
val CONVERT_METASTORE_ORC = buildConf("spark.sql.hive.convertMetastoreOrc")
.doc("When set to true, the built-in ORC reader and writer are used to
process " +
- "ORC tables created by using the HiveQL syntax, instead of Hive serde.")
+ "ORC tables created by using the HiveQL syntax, instead of Hive serde." +
+ "Enabling this parameter will cause insert overwrite when reading the
table," +
+ "and the file format of this table is ORC This error will be thrown[], "
+
Review comment:
Could you file a JIRA for this with your specific procedure? This one is
a valid bug.
> When the format of the table is parquet, there is no problem to insert and
overlay the same table, no matter whether this parameter is added or not【
spark.sql.hive .convertMetastoreParquet=false】
For this one, I think your statement in this PR is too broader because we
have a unit test coverage with Dynamic which is added by SPARK-30112. When you
say `this problem`, you should be specific to avoid ambiguity.
> [PartitionOverwriteMode.DYNAMIC] It can't solve the problem
> I use spark 3.0.0, spark-30112 does not solve this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
