Caideyipi commented on code in PR #17879:
URL: https://github.com/apache/iotdb/pull/17879#discussion_r3384904465
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AbstractMemTable.java:
##########
@@ -341,6 +337,38 @@ private static int computeTabletNullPointsNumber(
return nullPointsNumber;
}
+ private static int getValidMeasurementNumber(InsertRowNode insertNode,
boolean countFieldOnly) {
+ int count = 0;
+ for (int i = 0; i < insertNode.getMeasurements().length; i++) {
+ if (isValidMeasurement(insertNode, i, countFieldOnly)) {
+ count++;
+ }
+ }
+ return count;
+ }
+
+ private static int getValidMeasurementNumber(
+ InsertTabletNode insertNode, boolean countFieldOnly) {
+ int count = 0;
+ for (int i = 0; i < insertNode.getMeasurements().length; i++) {
+ if (isValidMeasurement(insertNode, i, countFieldOnly) &&
insertNode.getColumns()[i] != null) {
+ count++;
+ }
+ }
+ return count;
+ }
+
+ private static boolean isValidMeasurement(
+ InsertNode insertNode, int index, boolean countFieldOnly) {
+ return insertNode.getMeasurements()[index] != null
+ && (!countFieldOnly || isFieldMeasurement(insertNode, index));
+ }
+
+ private static boolean isFieldMeasurement(InsertNode insertNode, int index) {
+ return insertNode.getColumnCategories() == null
+ || insertNode.getColumnCategories()[index] ==
TsTableColumnCategory.FIELD;
+ }
Review Comment:
Done. Moved isValidMeasurement, isFieldMeasurement, and valid measurement
counting into InsertNode/InsertRowNode/InsertTabletNode. AbstractMemTable now
delegates to those helpers.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/InsertNode.java:
##########
@@ -189,13 +190,16 @@ public int measureColumnCnt() {
}
public boolean isValidMeasurement(int i) {
- return measurementSchemas != null
+ return measurements != null
Review Comment:
Good point. Normal insert paths should provide measurements; the null guard
here is defensive for partially rebuilt/deserialized nodes and existing
null-safe behavior when all measurements are cleared. I kept the check
defensive, and centralized the real validity rules in
InsertNode.isValidMeasurement(index, countFieldOnly).
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/InsertTabletNode.java:
##########
@@ -438,6 +438,16 @@ protected Object[] initTabletValues(int columnSize, int
rowSize, TSDataType[] da
return values;
}
+ protected Object[] initTabletValuesForSplit(int columnSize, int rowSize,
TSDataType[] dataTypes) {
+ Object[] values = initTabletValues(columnSize, rowSize, dataTypes);
+ for (int i = 0; i < values.length; i++) {
+ if (!hasColumnForSplit(i)) {
+ values[i] = null;
+ }
+ }
+ return values;
+ }
Review Comment:
Done. I changed the split path so initTabletValues delegates to a helper
with a split flag and only allocates arrays for columns that pass
hasColumnForSplit.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/MemUtils.java:
##########
@@ -67,14 +67,18 @@ public static long getRecordSize(TSDataType dataType,
Object value, boolean addi
* memory before insertion
*/
public static long getRowRecordSize(List<TSDataType> dataTypes, Object[]
value) {
- int emptyRecordCount = 0;
+ return getRowRecordSize(dataTypes, value, null);
+ }
+
+ public static long getRowRecordSize(
+ List<TSDataType> dataTypes, Object[] value, TsTableColumnCategory[]
columnCategories) {
+ int dataTypeIndex = 0;
long memSize = 0L;
for (int i = 0; i < value.length; i++) {
- if (value[i] == null) {
- emptyRecordCount++;
+ if (value[i] == null || !isFieldColumn(columnCategories, i)) {
continue;
}
- memSize += getRecordSize(dataTypes.get(i - emptyRecordCount), value[i],
false);
+ memSize += getRecordSize(dataTypes.get(dataTypeIndex++), value[i],
false);
}
return memSize;
}
Review Comment:
Done. Added comments in MemUtils clarifying that dataTypes contains only
non-null FIELD types and is ordered with the non-null FIELD values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
