JackieTien97 commented on PR #18748:
URL: https://github.com/apache/iotdb/pull/18748#issuecomment-5883059162
Reviewed commit `68302d6112902eccf3db6c66ea4604267eb21c5d`. I found two P2
issues that should be addressed before merging:
1. **[P2] Concurrent pipe deregistration can abort rebinding of the entire
metric set**
[PipeProcessorMetrics.java:109–126](https://github.com/apache/iotdb/blob/68302d6112902eccf3db6c66ea4604267eb21c5d/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/processor/PipeProcessorMetrics.java#L109-L126)
The new `unbindFrom()` passes a snapshot of task IDs directly to
`removeMetrics()`. If a task closes after that snapshot is taken,
`PipeProcessorSubtask.close()` can deregister it and remove it from
`processorMap` before the loop reaches its ID. `removeRate()` then reads `null`
and dereferences `processor.getPipeName()`.
Since `MetricService.restartService()` wraps both `unbindFrom()` and
`bindTo()` in the same `try`, this exception also skips rebinding for the
entire metric set. Other running pipes retain their registrations but lose
their exported processor metrics until another successful rebind.
I reproduced this with latches: pause the first snapshotted task during
unbinding, deregister the second task, then resume. Direct unbinding throws
`NullPointerException`; an actual `restartService()` leaves **0** processor
tablet-rate series for the remaining task instead of **1**. The same
direct-unbind test passes with the parent implementation, whose `deregister()`
path checks for already-removed IDs. This comparison isolates the newly
introduced exception; the parent still has the original restart-loss issue.
Please coordinate binding/unbinding with registration/removal, or
otherwise make the object lookup and removal safe against concurrent
deregistration. The other pipe/subscription metric sets changed to this
snapshot-and-remove pattern deserve the same check.
2. **[P2] Startup binds recovered dispatchers twice and overcounts timer
owners**
[LogDispatcherThreadMetrics.java:101–103](https://github.com/apache/iotdb/blob/68302d6112902eccf3db6c66ea4604267eb21c5d/iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/iot/logdispatcher/LogDispatcherThreadMetrics.java#L101-L103)
`REGION_TO_STAGE_TIMER_OWNERS` is incremented on every `bindStageTimer()`
call, but bindings are not always separated by an unbind.
`AbstractMetricService.addMetricSet()` binds immediately, even before the
service starts, and `startService()` binds all registered sets again.
This matches DataNode startup: `active()` restores the consensus layer
before `setUpMetricService()`, and the recovered IoTConsensus dispatchers
register their metric sets in their constructors. Each recovered dispatcher is
therefore counted twice. Removing all peers or deleting the region decrements
only once per dispatcher, leaving both `iot_send_log` stage timers exported and
phantom entries in the static owner map.
Reproduction: register two dispatcher metric sets before
`startService()`, start the service, then remove both. The assertion that the
last owner removes the timer fails on this PR and passes with the parent
version of `LogDispatcherThreadMetrics`. The existing test starts the service
before adding either dispatcher, so it misses this path.
Please make ownership registration idempotent, for example by tracking
owner instances, while still refreshing the cached timers on every bind.
Validation: a clean reactor test build of the 28 selected classes listed in
the PR description passed all **99 tests**. Three additional targeted tests
reproduced the findings above. The parent comparisons replaced only the
affected production classes (and the old pipe message constants), using the
same JVM, test classes, and dependency classpath; they were not full
parent-revision builds. No distributed end-to-end test was run for this review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]