Caideyipi opened a new pull request, #18830: URL: https://github.com/apache/iotdb/pull/18830
## Description An aligned tablet that introduces many measurements repeatedly copied full-width value, bitmap and accounting arrays while holding the DataRegion write lock. This backport registers missing measurements in insertion order and extends the TVList in one batch, resizing the reordered arrays once. Writes without schema extension keep the single-pass path, and new columns retain implicit nulls and lazy value blocks. Cherry-picks #18816 to `dev/1.3`, including both `1f05d5d3fb858519d580e4df6c1b766e47c03f63` and `5f27eb1ac3bc00f8568f672a14cc1ef9bdf581a3`. The backport uses the 1.3 measurement IDs, memchunk/tablet APIs and memory-accounting method. The upstream regression tests exposed two gaps in 1.3's handling of lazy blocks: an existing bitmap could hide a null value block, and WAL sizing/writing could dereference null Binary entries or unmaterialized arrays. Handle implicit nulls before consulting bitmaps and serialize null blocks with the existing value placeholders and null flags, preserving the WAL layout. A cross-block WAL round-trip test covers every supported scalar type and verifies the serialized byte count and that serialization keeps the source value blocks lazy. Regression coverage also includes existing/new/deleted and duplicate measurements, reordering with and without schema extension, nonzero tablet offsets, binary and primitive memory accounting, clone/clear/reuse, WAL recovery, handover, and a 10,000-column tablet followed by another write. ## Validation - `mvn spotless:apply -pl iotdb-core/datanode` - `mvn test -pl iotdb-core/datanode -am -Dtest=AlignedWritableMemChunkTest,AlignedTVListTest,MemChunkDeserializeTest,MemTableFlushTaskTest,PrimitiveMemTableTest,AlignedTVListIteratorTest -Dsurefire.failIfNoSpecifiedTests=false` — all 26 reactor modules built successfully; 67 tests passed in both configured Surefire executions with no failures, errors or skips. The initial run compiled DataNode and its dependencies from a fresh worktree; the final run recompiled the compatibility changes. - `git diff --check` This PR has: - [x] been self-reviewed, including the existing region locking and query/flush ownership rules. - [x] added Javadocs and comments explaining batch expansion and implicit nulls. - [x] added unit tests for the new paths and 1.3 compatibility handling. Key changed classes: `AlignedWritableMemChunk`, `AlignedTVList`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
