JingsongLi commented on code in PR #10851:
URL: https://github.com/apache/paimon/pull/10851#discussion_r4493672434
##########
paimon-core/src/main/java/org/apache/paimon/operation/NormalPartitionExpire.java:
##########
@@ -161,14 +160,15 @@ private List<Map<String, String>> doExpire(
List<List<String>> expiredPartValues = new
ArrayList<>(partitionEntries.size());
for (PartitionEntry partition : partitionEntries) {
Object[] array = strategy.convertPartition(partition.partition());
- // DATE values are normalized for time extraction; use the stored
names for dropping.
+ // DATE and TIMESTAMP values are normalized for time extraction;
use the stored
+ // names for dropping.
Map<String, String> partitionSpec =
commit.pathFactory()
.partitionComputer()
.generatePartValues(partition.partition());
int index = 0;
for (String value : partitionSpec.values()) {
- if (array[index] instanceof LocalDate) {
+ if (array[index] != null &&
strategy.isNormalizedForTimeExtraction(index)) {
array[index] = value;
Review Comment:
Rechecked b79af8f: the new binary-key deletion fixes the local path, and 156
expiration/extractor/commit tests pass with normal JDK 8 Maven checks. The
metastore path still has the same P1: `doBatchExpire` delegates names to
`partitionModification.dropPartitions`, which reaches
`HiveCatalog.dropPartitions -> super.dropPartitions ->
FileStoreCommit.dropPartitions` and reparses the key.
This reproduces using the PR's own new tests: set
`metastore.partitioned-table=true` in `createPartitionedTable`.
`testExpireTimestampPartitionNamedAsDefaultPartition` leaves the epoch row and
deletes NULL; both `testExpireDropsSelectedLocalZonedTimestampPartition` naming
modes leave 00:00Z and delete 05:00Z. These are actual
commit/expiration/readback tests using the existing catalog callback, not just
checks of returned partition names.
I also ran the epoch/NULL case with the three expiration production files
restored to merge-base d7b521d: it deletes epoch and retains NULL. The
identical data assertion fails on this head (only the returned-name assertion
is omitted in both runs because base reports ISO and head reports SQL). Please
propagate the exact selected typed key through the metastore-backed data
deletion path too, while retaining stored names for HMS metadata operations;
selecting the right key only in the local branch leaves this regression active.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]