LuciferYang commented on code in PR #11296:
URL: https://github.com/apache/paimon/pull/11296#discussion_r4652373155
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/execution/TruncatePaimonTableWithFilterExec.scala:
##########
@@ -36,37 +39,54 @@ case class TruncatePaimonTableWithFilterExec(
extends PaimonLeafV2CommandExec {
override def run(): Seq[InternalRow] = {
+ partitionPredicate match {
+ case Some(p) =>
+ table match {
+ case fileStoreTable: FileStoreTable =>
+ val matchedPartitions =
+
fileStoreTable.newSnapshotReader().withPartitionFilter(p).partitions()
+ if (matchedPartitions.isEmpty) {
+ withCommit(table)(_.commit(JCollections.emptyList()))
+ } else {
+ truncateMatchedPartitions(fileStoreTable, matchedPartitions)
+ }
+ case _ =>
+ throw new UnsupportedOperationException("Unsupported truncate
table")
+ }
+ case _ =>
+ withCommit(table)(_.truncateTable())
+ }
+ Nil
+ }
+
+ /**
+ * Removes the data of the matched partitions by their stored values: an
empty static overwrite. A
+ * spec rebuilt from a stored value does not always parse back to it, such
as a blank string,
+ * whose name is the default partition name.
+ */
+ private def truncateMatchedPartitions(
+ fileStoreTable: FileStoreTable,
+ partitions: JList[BinaryRow]): Unit = {
+ val target =
+ fileStoreTable.copy(Map(CoreOptions.DYNAMIC_PARTITION_OVERWRITE.key ->
"false").asJava)
+ val commit = target.newCommit(CoreOptions.createCommitUser(new
Options(target.options())))
+ try {
+ commit
+ .withOperation(Snapshot.Operation.TRUNCATE)
+ .withOverwriteStaticPartitions(partitions)
+ .commit(JCollections.emptyList[CommitMessage]())
Review Comment:
Fixed the same way as #11226: the exec now passes the DELETE's partition
predicate to a new internal
`TableCommitImpl.truncatePartitions(PartitionPredicate)`, which uses the same
`FileStoreCommitImpl.dropPartitions` overwrite as the spec overload and runs no
maintenance. The new test sets your TTL options and checks that the expired
`2020-01-01` partition survives `DELETE ... WHERE dt = '2099-01-01'`. The core
change is identical in both PRs, so they merge cleanly in either order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]