Yikf commented on code in PR #6082:
URL: https://github.com/apache/kyuubi/pull/6082#discussion_r1502192826
##########
extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/write/HiveWriteBuilder.scala:
##########
@@ -47,45 +41,22 @@ case class HiveWriteBuilder(
info,
hiveTableCatalog,
forceOverwrite,
- mergePartitionSpec(),
- ifPartitionNotExists)
+ dynamicPartitionSpec())
}
override def overwrite(filters: Array[Filter]): WriteBuilder = {
- filters match {
- case Array(AlwaysTrue) => // no partition, do nothing
- case _ => staticPartition = deduplicateFilters(filters)
- }
- overwriteDynamicPartitions()
+ forceOverwrite = true
Review Comment:
The current DSv2 SupportsOverwrite#overwrite is designed to filter out
(delete) data that matches deleteExpr when writer commit, But in the current
catalyst implementation, these filterExprs are static partition EqualNullSafe
expressions, KSHC does not currently require these processing filterExprs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
