github-actions[bot] commented on code in PR #66426:
URL: https://github.com/apache/doris/pull/66426#discussion_r3710564070
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -3720,17 +3720,18 @@ public void truncateTable(String dbName, String
tableName, PartitionNamesInfo pa
// which is the right behavior.
long oldPartitionId = entry.getValue();
long newPartitionId = oldToNewPartitionId.get(oldPartitionId);
+ DataProperty dataProperty =
copiedTbl.getPartitionInfo().getDataProperty(oldPartitionId);
Review Comment:
[P1] Reject stale cloud truncates before commit
On `CloudInternalCatalog`, this now lets the losing truncate get past the
missing old ID, build its replacement partitions, and call
`afterCreatePartitions`/`commitPartition` before the existing stale-ID check.
MetaService commit removes the PREPARED recycle keys (and, in versioned mode,
writes the partition indexes), but the subsequent `DdlException` only runs
`failedCleanCallback`, whose `deleteTablet` calls remove FE inverted-index
entries and never invoke `dropCloudPartition`. These IDs therefore remain
committed and unreferenced, and the table version is bumped, each time this
race occurs in cloud mode. The new test exercises only `InternalCatalog`.
Please reject the stale ID before creation/commit, or explicitly roll back the
committed cloud partitions on this failure path.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -3720,17 +3720,18 @@ public void truncateTable(String dbName, String
tableName, PartitionNamesInfo pa
// which is the right behavior.
long oldPartitionId = entry.getValue();
long newPartitionId = oldToNewPartitionId.get(oldPartitionId);
+ DataProperty dataProperty =
copiedTbl.getPartitionInfo().getDataProperty(oldPartitionId);
Partition newPartition =
createPartitionWithIndices(db.getId(), copiedTbl,
newPartitionId, entry.getKey(),
copiedTbl.getIndexIdToMeta(),
partitionsDistributionInfo.get(oldPartitionId),
-
copiedTbl.getPartitionInfo().getDataProperty(oldPartitionId),
+ dataProperty,
copiedTbl.getPartitionInfo().getReplicaAllocation(oldPartitionId), null /*
version info */,
copiedTbl.getCopiedBfColumns(), tabletIdSet,
copiedTbl.isInMemory(),
copiedTbl.getPartitionInfo().getTabletType(oldPartitionId),
-
olapTable.getPartitionInfo().getDataProperty(oldPartitionId).getStoragePolicy(),
+ dataProperty.getStoragePolicy(),
Review Comment:
[P1] Fence concurrent partition-policy changes
Suppose truncate snapshots policy P0, then `MODIFY PARTITION ...
storage_policy=P1` finishes while truncate is blocked in this unlocked window.
`updatePartitionsProperties` sends P1 only to the old tablet IDs, and
`modifyPartitionsProperty` installs a live `DataProperty` with P1. This line
still puts P0 into every replacement tablet's `CreateReplicaTask`; final
validation does not compare `DataProperty`, and `replacePartition` transfers
the live P1 metadata to the new ID. FE therefore reports P1 while the
replacement BE tablets retain P0, with no update task having targeted them.
Please fence/validate `DataProperty` under the final lock and fail plus clean
up if it changed (or otherwise propagate the update) rather than always using
the stale snapshot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
