Copilot commented on code in PR #18826:
URL: https://github.com/apache/iotdb/pull/18826#discussion_r4278968909
##########
iotdb-core/calc-commons/src/main/java/org/apache/iotdb/calc/transformation/dag/column/unary/scalar/DiffFunctionColumnTransformer.java:
##########
@@ -39,23 +37,46 @@ public class DiffFunctionColumnTransformer extends
UnaryColumnTransformer {
// indicate whether lastValue is null
private boolean lastValueIsNull = true;
+ // DIFF must advance its state once per input, even when several
masks/consumers request it.
+ private Column currentInputResult;
+
public DiffFunctionColumnTransformer(
Type returnType, ColumnTransformer childColumnTransformer, boolean
ignoreNull) {
super(returnType, childColumnTransformer);
this.ignoreNull = ignoreNull;
}
@Override
- public void evaluateWithSelection(boolean[] selection) {
- // The DIFF function depends on forward and backward rows and cannot be
short-circuited.
- boolean[] selectionCopy = new boolean[selection.length];
- Arrays.fill(selectionCopy, true);
- childColumnTransformer.evaluateWithSelection(selectionCopy);
- Column column = childColumnTransformer.getColumn();
- ColumnBuilder columnBuilder =
returnType.createColumnBuilder(column.getPositionCount());
- doTransform(column, columnBuilder, selection);
- initializeColumnCache(columnBuilder.build());
- childColumnTransformer.clearCache();
+ public void evaluate() {
+ initializeColumnCache(evaluateCurrentInput());
+ }
+
+ @Override
+ protected Column evaluateSelection(boolean[] selection) {
+ // DIFF depends on all preceding rows, including rows rejected by a
CASE/filter mask.
+ return evaluateCurrentInput();
+ }
+
+ private Column evaluateCurrentInput() {
+ if (currentInputResult == null) {
+ childColumnTransformer.tryEvaluate();
+ Column column = childColumnTransformer.getColumn();
+ ColumnBuilder builder =
returnType.createColumnBuilder(column.getPositionCount());
+ doTransform(column, builder);
+ currentInputResult = builder.build();
Review Comment:
`currentInputResult` keeps each DIFF result alive until the next input, even
after its ordinary cache is consumed. `FilterAndProjectOperator` still
estimates a unary chain as two columns and reports only its input operator's
retained memory. For example, eight nested DIFFs now retain eight distinct
result columns, but the projection's expression-memory estimate does not grow
with that nesting. A sum of independent DIFFs has the same missing-retention
problem. Account for these per-input results in both `calculateMaxPeekMemory()`
and `calculateRetainedSizeAfterCallingNext()`, deduplicating shared
transformers, and add a nested-DIFF accounting regression. Keep the result
retention itself: it is needed to prevent repeated state advancement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]