Copilot commented on code in PR #18826:
URL: https://github.com/apache/iotdb/pull/18826#discussion_r4269964996
##########
iotdb-core/calc-commons/src/main/java/org/apache/iotdb/calc/transformation/dag/column/binary/BinaryColumnTransformer.java:
##########
@@ -54,21 +57,14 @@ public void evaluate() {
}
@Override
- public void evaluateWithSelection(boolean[] selection) {
- leftTransformer.evaluateWithSelection(selection);
- // attention: get positionCount before calling getColumn
- int positionCount = leftTransformer.getColumnCachePositionCount();
- Column leftColumn = leftTransformer.getColumn();
- // When the rightTransformer has the same constant as the leftTransformer,
the leftTransformer's
- // cache will be cleared, so need to getColumn before evaluate
rightTransformer.
- rightTransformer.evaluateWithSelection(selection);
- Column rightColumn = rightTransformer.getColumn();
+ protected Column evaluateSelection(boolean[] selection) {
+ Column leftColumn = leftTransformer.getColumnWithSelection(selection);
+ Column rightColumn = rightTransformer.getColumnWithSelection(selection);
+ int positionCount = leftColumn.getPositionCount();
ColumnBuilder builder = returnType.createColumnBuilder(positionCount);
doTransform(leftColumn, rightColumn, builder, positionCount, selection);
- initializeColumnCache(builder.build());
- this.leftTransformer.clearCache();
- this.rightTransformer.clearCache();
+ return builder.build();
Review Comment:
This new selection path is not safe for every binary transformer:
`DiffColumnTransformer` inherits it, but its `doTransform(..., selection)`
deliberately processes every row and advances `lastValue`. Because this method
now returns an uncached partial result, two CASE branches or a filter plus
projection can evaluate dynamic-`ignoreNull` `diff()` twice for the same input
block and advance state twice. The stateful binary DIFF implementation needs
the same per-input full-result retention as `DiffFunctionColumnTransformer` (or
an equivalent override).
##########
iotdb-core/calc-commons/src/main/java/org/apache/iotdb/calc/transformation/dag/column/unary/scalar/DiffFunctionColumnTransformer.java:
##########
@@ -39,23 +37,46 @@ public class DiffFunctionColumnTransformer extends
UnaryColumnTransformer {
// indicate whether lastValue is null
private boolean lastValueIsNull = true;
+ // DIFF must advance its state once per input, even when several
masks/consumers request it.
+ private Column currentInputResult;
+
public DiffFunctionColumnTransformer(
Type returnType, ColumnTransformer childColumnTransformer, boolean
ignoreNull) {
super(returnType, childColumnTransformer);
this.ignoreNull = ignoreNull;
}
@Override
- public void evaluateWithSelection(boolean[] selection) {
- // The DIFF function depends on forward and backward rows and cannot be
short-circuited.
- boolean[] selectionCopy = new boolean[selection.length];
- Arrays.fill(selectionCopy, true);
- childColumnTransformer.evaluateWithSelection(selectionCopy);
- Column column = childColumnTransformer.getColumn();
- ColumnBuilder columnBuilder =
returnType.createColumnBuilder(column.getPositionCount());
- doTransform(column, columnBuilder, selection);
- initializeColumnCache(columnBuilder.build());
- childColumnTransformer.clearCache();
+ public void evaluate() {
+ initializeColumnCache(evaluateCurrentInput());
+ }
+
+ @Override
+ protected Column evaluateSelection(boolean[] selection) {
+ // DIFF depends on all preceding rows, including rows rejected by a
CASE/filter mask.
+ return evaluateCurrentInput();
+ }
+
+ private Column evaluateCurrentInput() {
+ if (currentInputResult == null) {
+ childColumnTransformer.tryEvaluate();
+ Column column = childColumnTransformer.getColumn();
+ ColumnBuilder builder =
returnType.createColumnBuilder(column.getPositionCount());
+ doTransform(column, builder);
+ currentInputResult = builder.build();
+ }
+ return currentInputResult;
+ }
+
+ @Override
+ protected void resetForNewInput() {
+ currentInputResult = null;
+ }
Review Comment:
`clearCache()` still inherits the base implementation, which only clears
`columnCache`; it does not clear `currentInputResult`. After a caller clears
this transformer and initializes a new input, `getColumnWithSelection()` can
therefore return the previous block's retained DIFF column even though the
ordinary cache was cleared. Override `clearCache()` to null this per-input
result (without resetting `lastValue`/`lastValueIsNull`), or route the
explicit-clear path through the same per-input cleanup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]