JackieTien97 commented on code in PR #18647:
URL: https://github.com/apache/iotdb/pull/18647#discussion_r4023182998


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/TVList.java:
##########
@@ -966,9 +966,10 @@ protected void skipToCurrentTimeRangeStartPosition() {
           
this.getQueryContext().getQueryStatistics().addFilteredRowsOfRowLevel(newIndex 
- index);
         }
         index = newIndex;
+        // If the cursor does not move, a duplicate-timestamp group prepared 
for the current
+        // position remains valid. Invalidate it only after the cursor 
actually advances.
+        probeNext = false;

Review Comment:
   **[P1] Invalidate non-aligned batch state before preserving `probeNext`**
   
   `newIndex == index` does not guarantee that `probeNext` still describes the 
current record. Non-aligned `TVListIterator.nextBatch()` advances `index` 
directly but does not clear `probeNext` before returning. With this change, 
switching through an empty fake page preserves that stale `true`, so subsequent 
point reads skip `prepareNext()` and can return duplicate timestamps or deleted 
points. This matters when an earlier non-overlapping page is read in batches 
and a later overlapping page is consumed through the point-reader path.
   
   For example, consider a single non-aligned `LongTVList` with these physical 
records, in write order:
   
   ```text
   index  time  value
   0      1     1
   1      100   2
   2      100   3
   ```
   
   Read `[1,33]` in batches, visit the empty page `[34,66]`, then read 
`[67,100]` as points:
   
   1. The first `hasNextBatch()` prepares index 0 and sets `probeNext = true`.
   2. `nextBatch()` emits `(1,1)`, advances to index 1, and stops because time 
100 exceeds the first page boundary. It leaves `probeNext = true`, although 
index 1 has not been prepared for point reading.
   3. For the empty page, binary search returns index 1 again. Previously, this 
branch cleared `probeNext`, allowing `hasNextBatch()` to prepare the time-100 
group and advance to its latest record, index 2. After this change, preparation 
is skipped and index 1 remains selected.
   4. When the last page is selected, time 100 is already inside its range, so 
the early return leaves the stale flag untouched. The first point read emits 
`(100,2)`; the next read also emits `(100,3)`.
   
   ```text
   Expected last-page output: [(100,3)]
   Parent implementation:    [(100,3)]
   This PR:                  [(100,2), (100,3)]
   ```
   
   The same stale flag bypasses deletion checks. My ASC and DESC deletion 
variants both passed with the parent implementation and returned an extra 
deleted point with this PR.
   
   Please clear `probeNext` after non-aligned `nextBatch()` advances the 
cursor, as the aligned batch implementation already does, and cover the 
batch-to-point transition while retaining these aligned regressions.
   
   <details>
   <summary>Minimal JUnit reproducer (imports omitted)</summary>
   
   ```java
   @Test
   public void testNonAlignedBatchToPointAfterEmptyPage() throws Exception {
     LongTVList list = LongTVList.newList();
     list.putLong(1, 1);
     list.putLong(100, 2);
     list.putLong(100, 3);
   
     MemPointIterator iterator =
         list.iterator(
             Ordering.ASC,
             list.rowCount(),
             null,
             Collections.emptyList(),
             0,
             TSEncoding.PLAIN,
             1024,
             null);
   
     iterator.setCurrentPageTimeRange(new TimeRange(1, 33));
     int firstPageRows = 0;
     while (iterator.hasNextBatch()) {
       firstPageRows += iterator.nextBatch().getPositionCount();
     }
     Assert.assertEquals(1, firstPageRows);
   
     iterator.setCurrentPageTimeRange(new TimeRange(34, 66));
     Assert.assertFalse(iterator.hasNextBatch());
   
     iterator.setCurrentPageTimeRange(new TimeRange(67, 100));
     List<String> result = new ArrayList<>();
     while (iterator.hasNextTimeValuePair()) {
       TimeValuePair pair = iterator.nextTimeValuePair();
       result.add(pair.getTimestamp() + ":" + pair.getValue().getLong());
     }
     Assert.assertEquals(Collections.singletonList("100:3"), result);
   }
   ```
   
   This minimal test supplies the page ranges explicitly. I also reproduced it 
with 300,000 physical rows: 299,998 records at time 1 followed by `(100,2)` and 
`(100,3)`. `ReadOnlyMemChunk` naturally generated `[1,33]`, `[34,66]`, and 
`[67,100]`; the same batch/batch/point sequence through 
`LazyMemVersionPageReader` and `PriorityMergeReader` returned `[100:2, 100:3]` 
on this PR and `[100:3]` with the parent implementation.
   
   </details>



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to