luoyuxia commented on code in PR #3630:
URL: https://github.com/apache/fluss/pull/3630#discussion_r3602175472
##########
fluss-server/src/main/java/org/apache/fluss/server/coordinator/AutoPartitionManager.java:
##########
@@ -525,8 +526,15 @@ private void dropPartitions(
dropIterator = entry.getValue().iterator();
}
+ boolean shouldRemoveEntry = true;
while (dropIterator.hasNext()) {
String partitionName = dropIterator.next();
+ if (isHistoricalPartitionName(
Review Comment:
Agreed, this is a real correctness issue. Auto-partition cleanup is
currently driven by retention time and does not coordinate with tiering
progress. Historical lookup exposes the problem as silently stale results, but
it does not introduce the underlying lifecycle gap.
More generally, the same behavior already affects active consumers: for
example, a partition may be deleted while a Flink job is still consuming it,
causing the remaining data to become unavailable.
Fixing this requires additional lifecycle coordination rather than a local
change here. We need to define and persist a deletion barrier, verify that
every bucket has been fully tiered, handle stalled or failed tiering, and
recover the pending deletion state after restarts.
This PR only changes historical lookup routing and preserves the historical
system partition during retention cleanup. I’ll create a separate issue to
track safe partition expiration and link it here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
