Caideyipi opened a new pull request, #18724:
URL: https://github.com/apache/iotdb/pull/18724

   ## Description
   
   ### Reduce duplicate timestamp flush overhead
   
   - Cache the unsorted value index once per row instead of resolving it 
repeatedly.
   - Reuse the null check already performed while selecting the last non-null 
value for a duplicate timestamp.
   - Replace per-column Pair<Long, Integer> state with primitive arrays, 
avoiding boxing in the hot loop.
   - Apply the same logic to both the normal aligned flush path and the 
deleted-measurement mapping path.
   
   The duplicate timestamp merge behavior is unchanged: each value column keeps 
the last non-null value at a timestamp.
   
   ### Tests
   
   - Added correctness coverage for duplicate timestamps with the full 
measurement set.
   - Added correctness coverage after deleting a measurement, exercising column 
remapping.
   - Added an opt-in manual performance UT. It stays skipped during normal test 
runs and can be enabled with:
   
       mvn test -pl iotdb-core/datanode 
-Dtest=AlignedFlushDuplicateTimeCheckPerformanceTest 
-Diotdb.aligned.flush.duplicate-check.perf.enabled=true
   
   Benchmark workload: 16,384 rows, 8 aligned columns, four rows per duplicated 
timestamp, 128 scans per round, five rounds.
   
   | Implementation | CPU per scan | Allocated per scan |
   | --- | ---: | ---: |
   | Legacy | 1.099 ms | 3,846,576 bytes |
   | Optimized | 0.366 ms | 128 bytes |
   
   The optimized loop used about 33.33% of the legacy CPU time in this targeted 
benchmark.
   
   Verification:
   
       mvn -nsu test -pl iotdb-core/datanode 
'-Dtest=MemTableFlushTaskTest,AlignedFlushDuplicateTimeCheckPerformanceTest'
   
   13 tests ran: 12 passed and the opt-in performance UT was skipped as 
expected.
   
   ---
   
   This PR has:
   
   - [x] been self-reviewed.
   - [x] added comments explaining the intent where the optimized duplicate 
merge is non-obvious.
   - [x] added or updated unit tests to cover the changed code paths.
   
   ---
   
   ##### Key changed/added classes in this PR
   
   - AlignedWritableMemChunk
   - MemTableFlushTaskTest
   - AlignedFlushDuplicateTimeCheckPerformanceTest
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to