Copilot commented on code in PR #19340:
URL: https://github.com/apache/iotdb/pull/19340#discussion_r4638609574
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueue.java:
##########
@@ -1093,6 +1093,9 @@ private boolean initPrefetchUnderInitializationLock(final
RegionProgress regionP
commitManager.isConfigNodeProgressAvailable(consumerGroupId,
topicName, consensusGroupId);
final boolean hasPersistedState =
commitManager.hasPersistedState(consumerGroupId, topicName,
consensusGroupId);
+ // Claim transient commit ownership and discard any stale in-flight state
left by a previous
+ // queue or deactivation cycle before snapshotting committed progress and
starting replay.
+ commitManager.claimTransientInFlightState(consumerGroupId, topicName,
consensusGroupId, this);
Review Comment:
`claimTransientInFlightState(...)` calls `getOrCreateState(...)`, which will
eagerly materialize a commit state even when no persisted state exists and the
queue may never produce mappings/commits. This can increase memory footprint
and do unnecessary initialization for cold/unused subscriptions. Since stale
transient state can only exist when a state is already present, consider using
an 'if present' variant here (or adding a manager method that claims/clears
only when state exists) to avoid creating empty states during initialization.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionCommitManager.java:
##########
@@ -2076,6 +2149,12 @@ protected boolean removeEldestEntry(final
Map.Entry<ProgressKey, Boolean> eldest
private final Map<Integer, NavigableSet<ProgressKey>>
committedPendingKeysByWriter =
new HashMap<>();
+ /**
+ * Active queue instance owning transient in-flight mappings so delayed
async cleanup or late
+ * prefetch/commit calls from a superseded queue cannot corrupt a
replacement queue.
+ */
+ private Object transientOwner;
Review Comment:
Storing the queue instance (`this`) as `transientOwner` keeps a strong
reference to the queue inside a long-lived commit state, which can retain
otherwise-GCable queue objects if cleanup/claim never runs again (e.g.,
abnormal termination with no replacement). Consider storing an opaque
lightweight token (e.g., a generated `long`/`UUID` held by the queue) instead
of the queue object itself, or using a `WeakReference`, so fencing doesn’t
create long-lived object retention.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionCommitManager.java:
##########
@@ -2084,6 +2163,59 @@ public ConsensusSubscriptionCommitState(
syncPersistedProgress();
}
+ public void clearTransientInFlightState() {
+ synchronized (this) {
+ clearTransientInFlightStateUnderLock();
+ transientOwner = null;
+ }
+ }
+
+ void claimTransientInFlightState(final Object owner) {
+ synchronized (this) {
+ clearTransientInFlightStateUnderLock();
+ transientOwner = owner;
+ }
+ }
+
+ boolean clearTransientInFlightStateIfOwner(final Object owner) {
+ synchronized (this) {
+ if (Objects.nonNull(owner) && Objects.nonNull(transientOwner) &&
transientOwner != owner) {
+ return false;
+ }
+ clearTransientInFlightStateUnderLock();
+ transientOwner = null;
+ return true;
+ }
+ }
Review Comment:
`clearTransientInFlightStateIfOwner` will clear state even when `owner ==
null` and `transientOwner` is set, which effectively bypasses fencing if a null
is accidentally passed through. To make fencing robust against accidental
misuse, consider treating `owner == null` as 'not authorized' when
`transientOwner != null` (return false), and keep
`clearTransientInFlightState()` as the explicit unfenced/force-clear API.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueue.java:
##########
@@ -3434,8 +3437,10 @@ private boolean createAndEnqueueEvent(
final SubscriptionCommitContext commitContext =
buildWriterCommitContext(commitLocalSeq);
final WriterId writerId = commitContext.getWriterId();
final WriterProgress writerProgress = commitContext.getWriterProgress();
- commitManager.recordMapping(
- consumerGroupId, topicName, consensusGroupId, writerId,
writerProgress);
+ if (!commitManager.recordMapping(
+ consumerGroupId, topicName, consensusGroupId, writerId,
writerProgress, this)) {
Review Comment:
When `recordMapping(...)` is rejected due to transient-owner fencing,
`createAndEnqueueEvent` returns `false` without any contextual logging, which
can make it hard to diagnose why prefetch stopped/enqueue failed during
failover/replacement scenarios. Consider adding a debug/trace log on the
rejection path (including consumerGroup/topic/region and whether the owner was
superseded) so operational triage can distinguish fencing from other enqueue
failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]