Caideyipi commented on PR #19021:
URL: https://github.com/apache/iotdb/pull/19021#issuecomment-6420983910

   Independent re-review of the complete diff and the related broker, consumer 
metadata, handshake, provider replacement, and ConfigNode procedure paths found 
a remaining **P1 concurrency gap between broker repair and new-region 
callbacks**.
   
   At the previous head, the [pre-setup queue 
snapshot](https://github.com/apache/iotdb/blob/0d2ef238dd6791ca04ed6bbd2bc13bec4825c9b1/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L425)
 could miss a queue created by a concurrent callback. Repair could then apply 
tail-proposal initialization to that queue, or its 
[rollback](https://github.com/apache/iotdb/blob/0d2ef238dd6791ca04ed6bbd2bc13bec4825c9b1/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L460)
 could remove the callback's queue and progress after another topic failed. 
Repair could also bind an already-visible new peer before its callback 
initialized it from the beginning, risking skipped initial writes.
   
   Fixed and pushed in **50008fb3179**:
   
   - [Serialize callbacks with consumer-group 
repair/rollback](https://github.com/apache/iotdb/blob/50008fb3179c9f3f525e722b3b352934e2c94306/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L199),
 re-read subscriptions after acquiring the lock, and preserve live queues 
instead of relying on the rollback snapshot.
   - [Mark new and recovered peers as callback-pending before 
publication](https://github.com/apache/iotdb/blob/50008fb3179c9f3f525e722b3b352934e2c94306/iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/iot/IoTConsensus.java#L365).
 [Existing-subscription repair defers those 
peers](https://github.com/apache/iotdb/blob/50008fb3179c9f3f525e722b3b352934e2c94306/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L650);
 fresh subscriptions still follow normal binding. Pending markers are cleared 
after callbacks, including failures.
   - Add regressions for callback lifecycle, repair during pending creation, 
fresh subscriptions, and a callback waiting across an unsubscribe; strengthen 
the live-queue initialization regression.
   
   Actual verification this round:
   
   - Local JDK 17: `mvn spotless:apply -P with-integration-tests -pl 
'iotdb-core/consensus,iotdb-core/datanode'` passed.
   - Remote Linux/JDK 17, separate pristine source trees and fresh isolated 
Maven repositories, build cache disabled, `-DforkCount=0`: **68 tests passed 
per locale, 0 failures/errors/skips**. The six selected suites were 
`IoTConsensusRecoveryTest`, `SubscriptionConsumerAgentTest`, 
`SubscriptionConsumerAgentBrokerRecoveryTest`, `SubscriptionReceiverV1Test`, 
`ConsensusSubscriptionSetupHandlerTest`, and 
`ConsensusSubscriptionCommitStateTest`.
   - The English whole-reactor `clean test` command ran those tests 
successfully but **ended with Maven exit 1** at `distribution`, which requires 
server/CLI/ConfigNode ZIPs that the `test` lifecycle does not produce. It is 
not a passing whole-reactor build.
   - The Chinese run used `clean test -pl '!distribution' -am -P 
with-zh-locale` with the same Surefire selection: **BUILD SUCCESS, 49 modules**.
   
   No integration tests or distribution packaging were run. This round's newly 
added tests were executed remotely, not locally.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to