CRZbulabula opened a new pull request, #18273:
URL: https://github.com/apache/iotdb/pull/18273
## Description
`CreateRegionGroupsProcedure` could race with `DeleteDatabaseProcedure`. If
the database partition table had already been removed when a replicated
`CreateRegionGroupsPlan` was applied, `PartitionInfo#createRegionGroups`
treated the database as not pre-deleted and dereferenced a missing
`DatabasePartitionTable`, causing an NPE in the ConfigRegion state machine.
Simply skipping a missing database is unsafe because Region replicas may
already have been created on DataNodes, and a delayed plan could be applied to
a later database that reuses the same name.
This PR:
- adds a monotonically increasing lifecycle generation for each database
incarnation and records it in RegionGroup allocation plans, procedure state,
and `PartitionInfo` snapshots;
- validates the complete `CreateRegionGroupsPlan` batch before mutation and
returns explicit statuses for missing, pre-deleted, or generation-mismatched
databases;
- serializes `CreateRegionGroupsProcedure` and `DeleteDatabaseProcedure`
against each other with database-level lifecycle locks, including atomic lock
acquisition for multi-database plans;
- submits independent `RemoveRegionGroupProcedure` compensation for replicas
that were created on DataNodes when RegionGroup persistence is rejected;
- advances the RegionGroup ID high-water mark even for rejected replicated
plans; and
- keeps snapshot and procedure deserialization compatible with data written
before lifecycle generations were introduced.
## Tests
The added tests cover:
- pre-deleted and fully removed databases;
- rejection of a stale plan after same-name database recreation;
- atomic validation of batched plans;
- RegionGroup ID advancement on rejected plans;
- snapshot, plan, and procedure serialization across recovery/leader changes;
- mutual exclusion between RegionGroup creation and database deletion; and
- compensation that removes only replicas successfully created on DataNodes.
Validation performed:
```text
mvn spotless:apply -pl iotdb-core/confignode
mvn test -pl iotdb-core/confignode
-Dtest=PartitionInfoTest,CreateRegionGroupsProcedureTest,ConfigPhysicalPlanSerDeTest
mvn compile -pl iotdb-core/confignode -DskipTests
mvn test-compile -DskipTests
mvn test-compile -P with-zh-locale -DskipTests
```
The targeted test run completed with 121 tests and no failures. Both
full-reactor locale builds completed successfully.
This PR has:
- [x] been self-reviewed.
- [x] concurrent write
- [x] added comments explaining non-obvious concurrency and recovery
behavior.
- [x] added or updated unit tests for the new code paths.
- [ ] added integration tests.
- [ ] been tested in a test IoTDB cluster.
##### Key changed/added classes
- `PartitionInfo`
- `DatabasePartitionTable`
- `CreateRegionGroupsPlan`
- `CreateRegionGroupsProcedure`
- `DeleteDatabaseProcedure`
- `AbstractDatabaseProcedure`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
