Caideyipi commented on PR #19021: URL: https://github.com/apache/iotdb/pull/19021#issuecomment-6420983910
Independent re-review of the complete diff and the related broker, consumer metadata, handshake, provider replacement, and ConfigNode procedure paths found a remaining **P1 concurrency gap between broker repair and new-region callbacks**. At the previous head, the [pre-setup queue snapshot](https://github.com/apache/iotdb/blob/0d2ef238dd6791ca04ed6bbd2bc13bec4825c9b1/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L425) could miss a queue created by a concurrent callback. Repair could then apply tail-proposal initialization to that queue, or its [rollback](https://github.com/apache/iotdb/blob/0d2ef238dd6791ca04ed6bbd2bc13bec4825c9b1/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L460) could remove the callback's queue and progress after another topic failed. Repair could also bind an already-visible new peer before its callback initialized it from the beginning, risking skipped initial writes. Fixed and pushed in **50008fb3179**: - [Serialize callbacks with consumer-group repair/rollback](https://github.com/apache/iotdb/blob/50008fb3179c9f3f525e722b3b352934e2c94306/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L199), re-read subscriptions after acquiring the lock, and preserve live queues instead of relying on the rollback snapshot. - [Mark new and recovered peers as callback-pending before publication](https://github.com/apache/iotdb/blob/50008fb3179c9f3f525e722b3b352934e2c94306/iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/iot/IoTConsensus.java#L365). [Existing-subscription repair defers those peers](https://github.com/apache/iotdb/blob/50008fb3179c9f3f525e722b3b352934e2c94306/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionSetupHandler.java#L650); fresh subscriptions still follow normal binding. Pending markers are cleared after callbacks, including failures. - Add regressions for callback lifecycle, repair during pending creation, fresh subscriptions, and a callback waiting across an unsubscribe; strengthen the live-queue initialization regression. Actual verification this round: - Local JDK 17: `mvn spotless:apply -P with-integration-tests -pl 'iotdb-core/consensus,iotdb-core/datanode'` passed. - Remote Linux/JDK 17, separate pristine source trees and fresh isolated Maven repositories, build cache disabled, `-DforkCount=0`: **68 tests passed per locale, 0 failures/errors/skips**. The six selected suites were `IoTConsensusRecoveryTest`, `SubscriptionConsumerAgentTest`, `SubscriptionConsumerAgentBrokerRecoveryTest`, `SubscriptionReceiverV1Test`, `ConsensusSubscriptionSetupHandlerTest`, and `ConsensusSubscriptionCommitStateTest`. - The English whole-reactor `clean test` command ran those tests successfully but **ended with Maven exit 1** at `distribution`, which requires server/CLI/ConfigNode ZIPs that the `test` lifecycle does not produce. It is not a passing whole-reactor build. - The Chinese run used `clean test -pl '!distribution' -am -P with-zh-locale` with the same Surefire selection: **BUILD SUCCESS, 49 modules**. No integration tests or distribution packaging were run. This round's newly added tests were executed remotely, not locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
