Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


JackieTien97 merged PR #18409:
URL: https://github.com/apache/iotdb/pull/18409


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


sonarqubecloud[bot] commented on PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#issuecomment-5248953883

   ## [![Quality Gate 
Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png
 'Quality Gate 
Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
**Quality Gate failed**  
   Failed conditions  
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png
 '') [C Reliability Rating on New 
Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
(required ≥ A)  
 
   
   
   [See analysis details on SonarQube 
Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409)
   
   ##   
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png
 '') Catch issues before they fail your Quality Gate with our IDE extension 
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png
 '') [SonarQube for 
IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


sonarqubecloud[bot] commented on PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#issuecomment-5248407670

   ## [![Quality Gate 
Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png
 'Quality Gate 
Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
**Quality Gate failed**  
   Failed conditions  
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png
 '') [C Reliability Rating on New 
Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
(required ≥ A)  
 
   
   
   [See analysis details on SonarQube 
Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409)
   
   ##   
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png
 '') Catch issues before they fail your Quality Gate with our IDE extension 
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png
 '') [SonarQube for 
IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


shuwenwei commented on code in PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#discussion_r3754863147


##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java:
##
@@ -212,39 +269,190 @@ public synchronized AlignedTVList cloneForFlushSort() {
   public synchronized AlignedTVList clone() {
 AlignedTVList cloneList = AlignedTVList.newAlignedList(new 
ArrayList<>(dataTypes));
 cloneAs(cloneList);
-cloneList.timeDeletedCnt = this.timeDeletedCnt;
-System.arraycopy(
-memoryBinaryChunkSize, 0, cloneList.memoryBinaryChunkSize, 0, 
dataTypes.size());
+cloneColumnDataTo(cloneList, null);
+cloneList.materializedValueArrayCounts =
+Arrays.copyOf(materializedValueArrayCounts, 
materializedValueArrayCounts.length);
+cloneList.materializedValueArrayMemCost = materializedValueArrayMemCost;
+return cloneList;
+  }
+
+  /**
+   * Prepare a partial clone without changing this TVList. The returned plan 
must be committed only
+   * after the query-memory reservation succeeds.
+   */
+  public synchronized PartialClonePlan preparePartialClone(Set 
columnsToClone) {
+Set retainedColumns =
+new HashSet<>(
+Objects.requireNonNull(
+columnsToClone,
+
DataNodeMiscMessages.EXCEPTION_COLUMNSTOCLONE_CANNOT_BE_NULL_458FDF37));
+AlignedTVList cloneList = AlignedTVList.newAlignedList(new 
ArrayList<>(dataTypes), false);
+// Pre-create the inner value lists for the retained columns; the other 
slots stay null until
+// the ownership transfer moves the source columns into place.
 for (int i = 0; i < values.size(); i++) {
-  // Clone value
+  if (retainedColumns.contains(i)) {
+cloneList.values.set(i, new ArrayList<>(values.get(i).size()));
+  }
+}
+cloneAs(cloneList);
+cloneColumnDataTo(cloneList, retainedColumns);
+return prepareMovePlan(cloneList, retainedColumns);
+  }
+
+  @SuppressWarnings("unchecked")
+  private PartialClonePlan prepareMovePlan(AlignedTVList cloneList, 
Set retainedColumns) {
+Objects.requireNonNull(
+cloneList, 
DataNodeMiscMessages.EXCEPTION_CLONELIST_CANNOT_BE_NULL_47AEEA8F);
+int columnCount = values.size();
+if (cloneList.values.size() != columnCount
+|| cloneList.memoryBinaryChunkSize.length != 
memoryBinaryChunkSize.length) {
+  throw new IllegalStateException(
+  DataNodeMiscMessages
+  
.EXCEPTION_TARGET_ALIGNEDTVLIST_HAS_INCOMPATIBLE_COLUMN_CONTAINERS_31FAC613);
+}
+
+List[] valueColumnsToMove = (List[]) new 
List[columnCount];
+List[] bitmapColumnsToMove = (List[]) new 
List[columnCount];

Review Comment:
   Fixed in a35f0763 by taking the "eliminate these arrays" option: 
`prepareMovePlan` now validates the move without allocating any O(N) structure 
(all existing checks are kept), and `commitPartialClone` derives the moved 
columns from the retained set (`retainedColumns.contains(i) == false`), so the 
commit is fully allocation-free.
   
   On the question of whether the two arrays deserve a separate reservation: 
after the earlier change (f7df7b689c) removed the N per-column inner ArrayLists 
by creating the clone with null outer slots and pre-creating inner lists only 
for retained columns, these two `columnCount` reference arrays (2 x N x 
NUM_BYTES_OBJECT_REF) were the only remaining transient allocation between the 
reservation and the commit. They are now gone entirely, so the preparation 
phase between `reserveMemoryCumulatively(listRamInfo)` and `commit()` is 
allocation-free and the admission-control claim holds by construction — there 
is no temporary O(N) state left to reserve or release. The existing 
partial-clone and end-to-end FragmentInstanceExecution tests exercise 
preparePartialClone + commit on sparse/wide layouts and pass.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


JackieTien97 commented on code in PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#discussion_r3749226841


##
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/utils/datastructure/AlignedTVListTest.java:
##
@@ -586,4 +589,166 @@ public void testCalculateChunkSize() {
 Assert.assertEquals(tvList.memoryBinaryChunkSize[0], 0);
 Assert.assertEquals(tvList.memoryBinaryChunkSize[1], 0);
   }
+
+  @Test
+  public void testMovesUnclonedColumns() {
+List dataTypes = new ArrayList<>();
+for (int i = 0; i < 3; i++) {
+  dataTypes.add(TSDataType.INT64);
+}
+AlignedTVList tvList = AlignedTVList.newAlignedList(dataTypes);
+tvList.putAlignedValue(0, new Object[] {1L, 2L, null});
+
+Set columnsToClone = Collections.singleton(1);
+long retainedRamSize = 
tvList.calculateRamSize(columnsToClone).getRamSize();
+AlignedTVList.PartialClonePlan partialClonePlan = 
tvList.preparePartialClone(columnsToClone);
+AlignedTVList clonedTvList = partialClonePlan.getCloneList();
+
+Assert.assertNotNull(tvList.getValues().get(0));
+Assert.assertNotNull(tvList.getValues().get(2));
+Assert.assertEquals(1L, tvList.getLongByValueIndex(0, 0));
+Assert.assertTrue(tvList.isNullValue(0, 2));
+Assert.assertEquals(2L, clonedTvList.getLongByValueIndex(0, 1));
+
+partialClonePlan.commit();
+
+Assert.assertNull(tvList.getValues().get(0));
+Assert.assertNull(tvList.getValues().get(2));
+Assert.assertTrue(tvList.isNullValue(0, 0));
+Assert.assertTrue(tvList.isNullValue(0, 2));
+Assert.assertEquals(1L, clonedTvList.getLongByValueIndex(0, 0));
+Assert.assertEquals(2L, clonedTvList.getLongByValueIndex(0, 1));
+Assert.assertTrue(clonedTvList.isNullValue(0, 2));
+Assert.assertEquals(retainedRamSize, 
tvList.calculateRamSize().getRamSize());
+  }
+
+  @Test
+  public void testPartialRamSizeScalesWithRetainedColumns() {
+int columnCount = 256;
+List dataTypes = new ArrayList<>(columnCount);
+Object[] values = new Object[columnCount];
+for (int i = 0; i < columnCount; i++) {
+  dataTypes.add(TSDataType.INT64);
+  values[i] = (long) i;
+}
+
+AlignedTVList tvList = AlignedTVList.newAlignedList(dataTypes);
+tvList.putAlignedValue(1, values);
+Set retainedColumns = Collections.singleton(0);
+long retainedRamSize = 
tvList.calculateRamSize(retainedColumns).getRamSize();
+long fullRamSize = tvList.calculateRamSize().getRamSize();
+
+// Only the retained column's materialized arrays are charged, so keeping 
1 of 256 columns
+// must cost far less than the full list.
+Assert.assertTrue(retainedRamSize < fullRamSize / 64);

Review Comment:
   **[P1] Restore the focused unit suite after changing the RAM model**
   
   At the latest head, `mvn test -pl iotdb-core/datanode -am 
-Dtest=AlignedTVListTest,FragmentInstanceExecutionTest 
-Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false` runs the 
end-to-end fragment tests successfully but fails 5 of the 23 
`AlignedTVListTest` cases. This assertion can no longer hold because the 
retained source intentionally keeps an N-wide fixed container baseline; four 
existing delta/absolute-size assertions likewise omit the newly added one-time 
container costs. Please update the affected invariants and expectations for the 
new model (including a fixed N-wide baseline plus the variable retained 
payload) and make the focused suite pass rather than weakening this ratio in 
isolation.



##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java:
##
@@ -212,39 +269,190 @@ public synchronized AlignedTVList cloneForFlushSort() {
   public synchronized AlignedTVList clone() {
 AlignedTVList cloneList = AlignedTVList.newAlignedList(new 
ArrayList<>(dataTypes));
 cloneAs(cloneList);
-cloneList.timeDeletedCnt = this.timeDeletedCnt;
-System.arraycopy(
-memoryBinaryChunkSize, 0, cloneList.memoryBinaryChunkSize, 0, 
dataTypes.size());
+cloneColumnDataTo(cloneList, null);
+cloneList.materializedValueArrayCounts =
+Arrays.copyOf(materializedValueArrayCounts, 
materializedValueArrayCounts.length);
+cloneList.materializedValueArrayMemCost = materializedValueArrayMemCost;
+return cloneList;
+  }
+
+  /**
+   * Prepare a partial clone without changing this TVList. The returned plan 
must be committed only
+   * after the query-memory reservation succeeds.
+   */
+  public synchronized PartialClonePlan preparePartialClone(Set 
columnsToClone) {
+Set retainedColumns =
+new HashSet<>(
+Objects.requireNonNull(
+columnsToClone,
+
DataNodeMiscMessages.EXCEPTION_COLUMNSTOCLONE_CANNOT_BE_NULL_458FDF37));
+AlignedTVList cloneList = AlignedTVList.newAlignedList(new 
ArrayList<>(dataTypes), false);
+// Pre-create the inner value lists for the retained columns; the other 
slots stay null until
+

Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


sonarqubecloud[bot] commented on PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#issuecomment-5239194997

   ## [![Quality Gate 
Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png
 'Quality Gate 
Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
**Quality Gate failed**  
   Failed conditions  
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png
 '') [C Reliability Rating on New 
Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
(required ≥ A)  
 
   
   
   [See analysis details on SonarQube 
Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409)
   
   ##   
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png
 '') Catch issues before they fail your Quality Gate with our IDE extension 
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png
 '') [SonarQube for 
IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


JackieTien97 commented on code in PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#discussion_r3748229489


##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java:
##
@@ -212,39 +260,165 @@ public synchronized AlignedTVList cloneForFlushSort() {
   public synchronized AlignedTVList clone() {
 AlignedTVList cloneList = AlignedTVList.newAlignedList(new 
ArrayList<>(dataTypes));
 cloneAs(cloneList);
-cloneList.timeDeletedCnt = this.timeDeletedCnt;
-System.arraycopy(
-memoryBinaryChunkSize, 0, cloneList.memoryBinaryChunkSize, 0, 
dataTypes.size());
-for (int i = 0; i < values.size(); i++) {
-  // Clone value
+cloneColumnDataTo(cloneList, null);
+cloneList.materializedValueArrayCounts =
+Arrays.copyOf(materializedValueArrayCounts, 
materializedValueArrayCounts.length);
+cloneList.materializedValueArrayMemCost = materializedValueArrayMemCost;
+return cloneList;
+  }
+
+  /**
+   * Prepare a partial clone without changing this TVList. The returned plan 
must be committed only
+   * after the query-memory reservation succeeds.
+   */
+  public synchronized PartialClonePlan preparePartialClone(Set 
columnsToClone) {
+Set retainedColumns =
+new HashSet<>(Objects.requireNonNull(columnsToClone, "columnsToClone 
cannot be null"));
+AlignedTVList cloneList = AlignedTVList.newAlignedList(new 
ArrayList<>(dataTypes));
+cloneAs(cloneList);
+cloneColumnDataTo(cloneList, retainedColumns);
+return prepareMovePlan(cloneList, retainedColumns);
+  }
+
+  @SuppressWarnings("unchecked")
+  private PartialClonePlan prepareMovePlan(AlignedTVList cloneList, 
Set retainedColumns) {
+Objects.requireNonNull(cloneList, "cloneList cannot be null");
+int columnCount = values.size();
+if (cloneList.values.size() != columnCount
+|| cloneList.memoryBinaryChunkSize.length != 
memoryBinaryChunkSize.length) {
+  throw new IllegalStateException("Target AlignedTVList has incompatible 
column containers");
+}
+
+List[] valueColumnsToMove = (List[]) new 
List[columnCount];

Review Comment:
   **[P1] Cover the O(N) partial-clone preparation peak before allocating it**
   
   The caller reserves only the final retained-source size before entering this 
method. However, `newAlignedList` has already created an inner `ArrayList` for 
every column (potentially preallocating `getDefaultArrayNum()` slots), and 
these two N-entry move arrays are allocated as additional preparation state. 
For every non-retained column, the clone's empty inner list is replaced during 
commit and becomes garbage, so this transient O(N) allocation is absent from 
both the final source size and the existing reservation. This remains true even 
after adding the retained-container cost requested in the earlier thread, so 
the claim that the transient clone increase is protected by admission control 
is not currently valid for wide aligned devices.
   
   Please either construct a partial clone with null outer slots and allocate 
inner lists only for retained columns, or estimate and reserve the preparation 
overhead separately and release that temporary reservation after 
commit/rollback. Add a wide sparse-column test that exercises this 
peak-accounting path.



##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java:
##
@@ -1137,22 +1410,51 @@ public synchronized RamInfo calculateRamSize() {
 new ArrayList<>(dataTypes));
   }
 
+  public synchronized RamInfo calculateRamSize(Set columnsToClone) {
+return new RamInfo(
+timestamps.size(),
+alignedTvListArrayMemCost(columnsToClone),
+getRamSize(columnsToClone),
+rowCount,
+new ArrayList<>(dataTypes));
+  }
+
   public synchronized long getRamSize() {
 return (long) timestamps.size() * 
alignedTvListArrayMemCostWithoutPrimitiveArrays()
 + materializedValueArrayMemCost
 + materializedBitmapMemoryCost;
   }
 
-  private static long calculateBitmapRamCost(List> bitMaps) {
+  public synchronized long getRamSize(Set columnsToClone) {
+long size =
+(long) timestamps.size() * 
alignedTvListArrayMemCostWithoutPrimitiveArrays(columnsToClone);
+for (int i = 0; i < dataTypes.size(); i++) {
+  if (columnsToClone != null && !columnsToClone.contains(i)) {
+continue;
+  }
+  TSDataType dataType = dataTypes.get(i);
+  if (dataType != null) {
+size += (long) materializedValueArrayCounts[i] * 
valueListArrayMemCost(dataType);

Review Comment:
   **[P1] Count lazy null backing slots independently of materialized arrays**
   
   On master, a retained value column can have B block entries in its inner 
`ArrayList` but only M materialized primitive arrays. This line charges 
`NUM_BYTES_OBJECT_REF` only M times via `valueListArrayMemCost`; meanwhile 
`alignedTvListArrayMemCostWithoutPrimitiveArrays` 

Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


JackieTien97 commented on code in PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#discussion_r3747590327


##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AbstractWritableMemChunk.java:
##
@@ -109,19 +111,39 @@ protected void maybeReleaseTvList(TVList tvList) {
 }
   }
 
+  /**
+   * Try to release the TVList. If there are active queries, transfer memory 
ownership to the first
+   * query. For AlignedTVList, this will release non-query columns before 
transferring to reduce
+   * memory footprint.
+   */
   private void tryReleaseTvList(TVList tvList) {
-long tvListRamSize = tvList.calculateRamSize().getRamSize();
 tvList.lockQueryList();
 try {
   if (tvList.getQueryContextSet().isEmpty()) {
 tvList.clear();
   } else {
 QueryContext firstQuery = 
tvList.getQueryContextSet().iterator().next();
+
+// For AlignedTVList with active queries, release non-query columns 
before
+// transferring memory ownership to reduce memory footprint.
+if (tvList instanceof AlignedTVList) {
+  AlignedTVList alignedTVList = (AlignedTVList) tvList;
+
+  // Get the union of all columns accessed by queries
+  Set accessedColumns = 
alignedTVList.getAccessedColumnsForQuery();
+
+  if (accessedColumns != null && !accessedColumns.isEmpty()) {

Review Comment:
   **[P2] Preserve the meaning of a tracked empty column set**
   
   A table-model time-only scan legitimately records an empty 
`columnIndexList`. This guard, together with 
`AlignedTVList.releaseNonQueryColumns` returning early for an empty set, keeps 
every value array when the flushed TVList is transferred to that query. The 
most selective case therefore retains the full wide aligned list and defeats 
this PR's memory-reduction goal.
   
   Please distinguish "tracked and empty" (time-only: release all value 
columns) from "not tracked/unsupported context" (unknown: conservatively retain 
all columns), and add a focused table-model time-only ownership-transfer test.



##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java:
##
@@ -1137,22 +1410,51 @@ public synchronized RamInfo calculateRamSize() {
 new ArrayList<>(dataTypes));
   }
 
+  public synchronized RamInfo calculateRamSize(Set columnsToClone) {
+return new RamInfo(
+timestamps.size(),
+alignedTvListArrayMemCost(columnsToClone),
+getRamSize(columnsToClone),
+rowCount,
+new ArrayList<>(dataTypes));
+  }
+
   public synchronized long getRamSize() {
 return (long) timestamps.size() * 
alignedTvListArrayMemCostWithoutPrimitiveArrays()
 + materializedValueArrayMemCost
 + materializedBitmapMemoryCost;
   }
 
-  private static long calculateBitmapRamCost(List> bitMaps) {
+  public synchronized long getRamSize(Set columnsToClone) {
+long size =
+(long) timestamps.size() * 
alignedTvListArrayMemCostWithoutPrimitiveArrays(columnsToClone);
+for (int i = 0; i < dataTypes.size(); i++) {
+  if (columnsToClone != null && !columnsToClone.contains(i)) {
+continue;
+  }
+  TSDataType dataType = dataTypes.get(i);
+  if (dataType != null) {
+size += (long) materializedValueArrayCounts[i] * 
valueListArrayMemCost(dataType);
+  }
+}
+return size + calculateBitmapRamCost(bitMaps, columnsToClone);

Review Comment:
   **[P1] Include the retained wide-column containers in query memory 
accounting**
   
   The final follow-up in #18394 (`c4f31171`) added `calculateContainerRamCost` 
because, after a partial move, the query-owned source TVList still retains 
N-wide structures such as `dataTypes`, `memoryBinaryChunkSize`, and the outer 
`values`/`bitMaps` containers. On master it also retains the N-wide 
`materializedValueArrayCounts` array. This implementation charges only 
timestamps, materialized value arrays, and bitmaps, so a query touching one 
column of a very wide aligned device can under-reserve O(N) memory even though 
the clone path claims its transient allocation is protected by admission 
control.
   
   Please port that final accounting fix, adapted to master's lazy-allocation 
fields, and restore an equivalent wide-container assertion (plus any affected 
RAM expectations). Otherwise this is not semantically equivalent to the final 
#18394 implementation. Reference: 
https://github.com/apache/iotdb/commit/c4f31171b69d38f42ae5c39c29226438dccec03f



##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java:
##
@@ -212,39 +260,165 @@ public synchronized AlignedTVList cloneForFlushSort() {
   public synchronized AlignedTVList clone() {
 AlignedTVList cloneList = AlignedTVList.newAlignedList(new 
ArrayList<>(dataTypes));
 cloneAs(cloneList);
-cloneList.timeDeletedCnt = this.timeDeletedCnt;
-System.arraycopy(
-  

Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-10 Thread via GitHub


JackieTien97 commented on code in PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#discussion_r3747518209


##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/fragment/FragmentInstanceContext.java:
##
@@ -1131,6 +1178,7 @@ public synchronized void releaseResource() {
 
 // release TVList/AlignedTVList owned by current query
 releaseTVListOwnedByQuery();
+alignedTVListColumnAccessMap.clear();

Review Comment:
   ```suggestion
   alignedTVListColumnAccessMap = null;
   ```
   
   set to null may be better and more effecient



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-09 Thread via GitHub


codecov[bot] commented on PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#issuecomment-5235638533

   ## 
[Codecov](https://app.codecov.io/gh/apache/iotdb/pull/18409?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 Report
   :x: Patch coverage is `77.47748%` with `75 lines` in your changes missing 
coverage. Please review.
   :white_check_mark: Project coverage is 43.64%. Comparing base 
([`7e2c453`](https://app.codecov.io/gh/apache/iotdb/commit/7e2c453fbd1880e977c4da50395881a36edc1898?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache))
 to head 
([`6be2c09`](https://app.codecov.io/gh/apache/iotdb/commit/6be2c09eef59dfd1dc8e724d19726ec318aadbd1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 8 commits behind head on master.
   
   | [Files with missing 
lines](https://app.codecov.io/gh/apache/iotdb/pull/18409?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 | Patch % | Lines |
   |---|---|---|
   | 
[...engine/schemaregion/utils/ResourceByPathUtils.java](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fschemaengine%2Fschemaregion%2Futils%2FResourceByPathUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc2NoZW1hZW5naW5lL3NjaGVtYXJlZ2lvbi91dGlscy9SZXNvdXJjZUJ5UGF0aFV0aWxzLmphdmE=)
 | 66.00% | [34 Missing :warning: 
](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 |
   | 
[...he/iotdb/db/utils/datastructure/AlignedTVList.java](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Futils%2Fdatastructure%2FAlignedTVList.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvdXRpbHMvZGF0YXN0cnVjdHVyZS9BbGlnbmVkVFZMaXN0LmphdmE=)
 | 90.15% | [19 Missing :warning: 
](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 |
   | 
[...ne/execution/fragment/FragmentInstanceContext.java](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fexecution%2Ffragment%2FFragmentInstanceContext.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvZXhlY3V0aW9uL2ZyYWdtZW50L0ZyYWdtZW50SW5zdGFuY2VDb250ZXh0LmphdmE=)
 | 16.66% | [15 Missing :warning: 
](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 |
   | 
[...ner/memory/ThreadSafeMemoryReservationManager.java](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Fplanner%2Fmemory%2FThreadSafeMemoryReservationManager.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9wbGFubmVyL21lbW9yeS9UaHJlYWRTYWZlTWVtb3J5UmVzZXJ2YXRpb25NYW5hZ2VyLmphdmE=)
 | 0.00% | [2 Missing :warning: 
](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 |
   | 
[...e/dataregion/memtable/AlignedReadOnlyMemChunk.java](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fmemtable%2FAlignedReadOnlyMemChunk.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL21lbXRhYmxlL0FsaWduZWRSZWFkT25seU1lbUNodW5rLmphdmE=)
 | 0.00% | [2 Missing :warning: 
](https://app.codecov.io/gh/apache/iotdb/pull/18409?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 |
   | 

Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-09 Thread via GitHub


sonarqubecloud[bot] commented on PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#issuecomment-5235397533

   ## [![Quality Gate 
Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png
 'Quality Gate 
Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
**Quality Gate failed**  
   Failed conditions  
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png
 '') [C Reliability Rating on New 
Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
(required ≥ A)  
 
   
   
   [See analysis details on SonarQube 
Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409)
   
   ##   
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png
 '') Catch issues before they fail your Quality Gate with our IDE extension 
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png
 '') [SonarQube for 
IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-07 Thread via GitHub


sonarqubecloud[bot] commented on PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#issuecomment-5215870066

   ## [![Quality Gate 
Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png
 'Quality Gate 
Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
**Quality Gate failed**  
   Failed conditions  
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png
 '') [C Reliability Rating on New 
Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
(required ≥ A)  
 
   
   
   [See analysis details on SonarQube 
Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409)
   
   ##   
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png
 '') Catch issues before they fail your Quality Gate with our IDE extension 
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png
 '') [SonarQube for 
IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] clone partial columns of aligned tvlist for query [iotdb]

2026-08-07 Thread via GitHub


sonarqubecloud[bot] commented on PR #18409:
URL: https://github.com/apache/iotdb/pull/18409#issuecomment-5214256713

   ## [![Quality Gate 
Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png
 'Quality Gate 
Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
**Quality Gate failed**  
   Failed conditions  
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png
 '') [C Reliability Rating on New 
Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409) 
(required ≥ A)  
 
   
   
   [See analysis details on SonarQube 
Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18409)
   
   ##   
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png
 '') Catch issues before they fail your Quality Gate with our IDE extension 
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png
 '') [SonarQube for 
IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]