VGalaxies opened a new pull request, #19339: URL: https://github.com/apache/iotdb/pull/19339
## Problem When an `AlignedTVList` receives out-of-order writes and is subsequently queried before flushing, query execution calls `AlignedTVList.sort()`, which allocates `indices` and `segmentMovedMap` without charging the owning `IMemTable`'s `structureMemoryBlock`. On a subsequent out-of-order write to the same working `AlignedTVList`: 1. `TsFileProcessor` previously measured live `getStructureRamSize()` **before** the write (which included the unaccounted `segmentMovedMap` bytes allocated by query `sort()`). 2. The out-of-order write cleared `segmentMovedMap`, causing post-write `getStructureRamSize()` to drop below the pre-write snapshot. 3. Reconciling the negative delta against `structureMemoryBlock` released bytes that had never been charged, triggering a `WriteMemoryBlock` underflow (`IllegalArgumentException` / failed flush or insert). The same gap also applied when query-side clone-on-write replaced the working TVList via `AlignedTVList.clone()` or `AlignedTVList.commitPartialClone(PartialClonePlan)` without preserving the accounted RAM baseline on the replacement working TVList. ## Fix - Track the last array and structure RAM sizes reconciled into the owning `IMemTable`'s `WriteMemoryBlock`s (`ramSizeAccounted`, `accountedArrayRamSize`, `accountedStructureRamSize`) on `AlignedTVList`. - Initialize/reset this baseline on WAL deserialization (`AlignedTVList.deserialize`) and `clear()`, and preserve it across full and partial clone replacements (`clone()` and `commitPartialClone(...)`). - Update `TsFileProcessor` RAM reconciliation for aligned TVLists to diff post-write actual RAM against the previously accounted baseline rather than a live pre-write snapshot that may include query-allocated structures. ## Tests - `AlignedTVListTest`: added `testAccountedRamSizeLifecycleOnDeserializeAndClear` and `testAccountedRamSizePreservedAcrossCloneAndPartialClone` (31/31 pass). - `TsFileProcessorTest`: added `alignedStructureMemoryAccountingSurvivesRepeatedSortAndOutOfOrderWrites`, `alignedStructureMemoryAccountingSurvivesMultiDeviceSortAndOutOfOrderBatch`, and `alignedStructureMemoryAccountingSurvivesCloneAndPartialCloneReplacement` (33/33 pass). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
