luoluoyuyu opened a new pull request, #19310:
URL: https://github.com/apache/iotdb/pull/19310

   ## Summary
   
   This PR reduces the write-path cost of aligned (multi-measurement) 
timeseries on devices with many columns. The previous code made several per-row 
operations scale with the device-wide column count , even when an insert 
touched one measurement.
   
   - **Sparse aligned writes:**  now passes a compact column-index mapping () 
to  instead of materializing device-wide  /  reorder arrays. Value copying and 
bitmap masking iterate only the columns in the insert plan. Columns that 
already hold a value array in the target block are still explicitly null-marked 
for the rows this write does not touch.
   - **No per-row schema stream:**  builds its filtered schema list only inside 
the  lambda, so existing devices no longer pay a stream/lambda allocation per 
insert.
   - **Cached structure RAM cost:**  caches  and invalidates it on structural 
changes (, , , , , , clone/move, sort metadata).  is now O(1) on the common 
path instead of two O(C) estimates per row.
   - **Lazy null prefix for new columns:** newly added value columns use a  
that represents existing blocks as implicit nulls and only materializes a block 
when it is first written.
   
   ## Example
   
   For a device with 1000 measurements, writing one row with one measurement:
   
   - Before: each row allocated  and  (about 16 KB of reference arrays), then 
scanned all 1000 device columns twice for value copy/null marking.
   - After: the write allocates ,  and , and copies/masks only the written 
column. The first  after a structural change still computes the full cost, but 
subsequent calls in the same write return the cached value.
   
   Behavior is unchanged: unwritten measurements remain null for the new row, 
including when their value arrays were materialized earlier in the same block.
   
   ## Validation
   
   ```bash
   mvn -pl iotdb-core/datanode -am \
     
-Dtest=AlignedTVListTest,AlignedWritableMemChunkTest,AlignedTVListIteratorTest \
     -Dsurefire.failIfNoSpecifiedTests=false test
   ```
   
   Result: 51 tests run, 0 failures, 0 errors.
   
   New tests cover sparse column mapping with bitmaps/results across a block 
boundary, structure-cost cache invalidation, lazy prefixes over multiple 
blocks, and single-measurement writes to a 10,000-column device.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to