luoluoyuyu opened a new pull request, #19310:
URL: https://github.com/apache/iotdb/pull/19310
## Summary
This PR reduces the write-path cost of aligned (multi-measurement)
timeseries on devices with many columns. The previous code made several per-row
operations scale with the device-wide column count , even when an insert
touched one measurement.
- **Sparse aligned writes:** now passes a compact column-index mapping ()
to instead of materializing device-wide / reorder arrays. Value copying and
bitmap masking iterate only the columns in the insert plan. Columns that
already hold a value array in the target block are still explicitly null-marked
for the rows this write does not touch.
- **No per-row schema stream:** builds its filtered schema list only inside
the lambda, so existing devices no longer pay a stream/lambda allocation per
insert.
- **Cached structure RAM cost:** caches and invalidates it on structural
changes (, , , , , , clone/move, sort metadata). is now O(1) on the common
path instead of two O(C) estimates per row.
- **Lazy null prefix for new columns:** newly added value columns use a
that represents existing blocks as implicit nulls and only materializes a block
when it is first written.
## Example
For a device with 1000 measurements, writing one row with one measurement:
- Before: each row allocated and (about 16 KB of reference arrays), then
scanned all 1000 device columns twice for value copy/null marking.
- After: the write allocates , and , and copies/masks only the written
column. The first after a structural change still computes the full cost, but
subsequent calls in the same write return the cached value.
Behavior is unchanged: unwritten measurements remain null for the new row,
including when their value arrays were materialized earlier in the same block.
## Validation
```bash
mvn -pl iotdb-core/datanode -am \
-Dtest=AlignedTVListTest,AlignedWritableMemChunkTest,AlignedTVListIteratorTest \
-Dsurefire.failIfNoSpecifiedTests=false test
```
Result: 51 tests run, 0 failures, 0 errors.
New tests cover sparse column mapping with bitmaps/results across a block
boundary, structure-cost cache invalidation, lazy prefixes over multiple
blocks, and single-measurement writes to a 10,000-column device.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]