JackieTien97 commented on code in PR #18875:
URL: https://github.com/apache/iotdb/pull/18875#discussion_r4282583202
##########
integration-test/src/test/java/org/apache/iotdb/db/it/query/IoTDBCaseWhenThenIT.java:
##########
@@ -928,4 +928,20 @@ public void testThenWithBinarySameConstant() {
new String[] {"0,0.0,", "1000000,11.0,", "20000000,22.0,",
"210000000,33.0,"};
resultSetEqualTest(sql, expectedHeader, retArray);
}
+
+ @Test
+ public void testNullableAndInLaterCaseBranch() {
+ prepareData(
+ new String[] {
+ "CREATE TIMESERIES root.sg.case_null.s1 WITH DATATYPE=INT32,
ENCODING=PLAIN",
+ "CREATE TIMESERIES root.sg.case_null.s2 WITH DATATYPE=INT32,
ENCODING=PLAIN",
+ "INSERT INTO root.sg.case_null(time, s2) VALUES (1, 1)",
+ "INSERT INTO root.sg.case_null(time, s1, s2) VALUES (2, 1, 1), (3,
0, 1), (4, 101, 1)"
+ });
+ resultSetEqualTest(
+ "SELECT CASE WHEN s1 > 100 THEN 1 WHEN s1 > 0 AND s2 > 0 THEN 2 ELSE 3
END AS result "
Review Comment:
Since the AND only appears as a WHEN condition, NULL and FALSE both mean "no
match", so this query only covers the `NULL AND TRUE` half of the fix (time 1:
`2.0` before the fix, `3.0` after). Time 3 (`0 > 0 AND 1 > 0`) is `3.0` whether
the AND returns FALSE or NULL: with line 73 of `LogicAndColumnTransformer`
changed to `builder.appendNull()`, this query still returns `3.0, 2.0, 3.0,
1.0`.
A query that exposes the AND result itself would cover the FALSE half too.
On the same data:
| query | before fix | after fix |
|---|---|---|
| `CASE WHEN s1 > 100 THEN 1 WHEN NOT (s1 > 0 AND s2 > 0) THEN 2 ELSE 3 END`
| `3.0, 3.0, 3.0, 1.0` | `3.0, 3.0, 2.0, 1.0` |
| `CASE WHEN s1 > 100 THEN false WHEN s2 > 0 THEN (s1 > 0 AND s2 > 0) END` |
`true, true, null, false` | `null, true, false, false` |
Both also fail with the `appendNull()` change (time 3 becomes `3.0` /
`null`). I got these values by running the expressions through
`ColumnTransformerVisitor` + `FilterAndProjectOperator` on this PR's code, with
and without the fix.
##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/transformation/dag/column/binary/LogicBinaryColumnTransformerTest.java:
##########
@@ -113,4 +113,60 @@ public void testInputType() {
}
}
+
+ @Test
+ public void testLogicAndWithNullsAndSelection() {
+ assertSelectedAnd(false);
+ }
+
+ @Test
+ public void testLogicAndWithNullsAndPartialSelection() {
+ assertSelectedAnd(true);
+ }
+
+ private void assertSelectedAnd(boolean partial) {
+ Boolean[] values = {null, false, true};
+ Boolean[] expected = {null, false, null, false, false, false, null, false,
true};
+ TsBlockBuilder builder =
+ new TsBlockBuilder(9, Arrays.asList(TSDataType.BOOLEAN,
TSDataType.BOOLEAN));
+ for (Boolean left : values) {
+ for (Boolean right : values) {
+ builder.getTimeColumnBuilder().writeLong(0);
+ if (left == null) {
+ builder.getColumnBuilder(0).appendNull();
+ } else {
+ builder.getColumnBuilder(0).writeBoolean(left);
+ }
+ if (right == null) {
+ builder.getColumnBuilder(1).appendNull();
+ } else {
+ builder.getColumnBuilder(1).writeBoolean(right);
+ }
+ builder.declarePosition();
+ }
+ }
+ TsBlock block = builder.build();
+ IdentityColumnTransformer left = new IdentityColumnTransformer(returnType,
0);
+ IdentityColumnTransformer right = new
IdentityColumnTransformer(returnType, 1);
+ left.addReferenceCount();
+ right.addReferenceCount();
+ left.initFromTsBlock(block);
+ right.initFromTsBlock(block);
+ LogicAndColumnTransformer transformer = new
LogicAndColumnTransformer(returnType, left, right);
Review Comment:
The bug came from `LogicAndColumnTransformer` and `LogicOrColumnTransformer`
drifting apart (OR already ends `evaluateWithSelection` with
`builder.build()`), but this helper hard-codes AND. So OR's
`evaluateWithSelection` still has no NULL/selection coverage: `testLogicOr`
only goes through `evaluate()`, and this is the only unit test that uses
`LogicOrColumnTransformer`.
Could the helper take the constructor and the expected table, so it runs for
both? Something like:
```java
assertSelected(
(l, r) -> new LogicAndColumnTransformer(returnType, l, r),
new Boolean[] {null, false, null, false, false, false, null, false,
true});
assertSelected(
(l, r) -> new LogicOrColumnTransformer(returnType, l, r),
new Boolean[] {null, null, true, null, false, true, true, true, true});
```
The OR table passes on the current code (full and partial selection), and
fails if OR picks up the same `initializeColumnCache(rightColumn)` bug.
##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/transformation/dag/column/binary/LogicBinaryColumnTransformerTest.java:
##########
@@ -113,4 +113,60 @@ public void testInputType() {
}
}
+
+ @Test
+ public void testLogicAndWithNullsAndSelection() {
+ assertSelectedAnd(false);
+ }
+
+ @Test
+ public void testLogicAndWithNullsAndPartialSelection() {
+ assertSelectedAnd(true);
+ }
+
+ private void assertSelectedAnd(boolean partial) {
+ Boolean[] values = {null, false, true};
+ Boolean[] expected = {null, false, null, false, false, false, null, false,
true};
+ TsBlockBuilder builder =
+ new TsBlockBuilder(9, Arrays.asList(TSDataType.BOOLEAN,
TSDataType.BOOLEAN));
+ for (Boolean left : values) {
+ for (Boolean right : values) {
+ builder.getTimeColumnBuilder().writeLong(0);
+ if (left == null) {
+ builder.getColumnBuilder(0).appendNull();
+ } else {
+ builder.getColumnBuilder(0).writeBoolean(left);
+ }
+ if (right == null) {
+ builder.getColumnBuilder(1).appendNull();
+ } else {
+ builder.getColumnBuilder(1).writeBoolean(right);
+ }
+ builder.declarePosition();
+ }
+ }
+ TsBlock block = builder.build();
+ IdentityColumnTransformer left = new IdentityColumnTransformer(returnType,
0);
+ IdentityColumnTransformer right = new
IdentityColumnTransformer(returnType, 1);
+ left.addReferenceCount();
+ right.addReferenceCount();
+ left.initFromTsBlock(block);
+ right.initFromTsBlock(block);
+ LogicAndColumnTransformer transformer = new
LogicAndColumnTransformer(returnType, left, right);
+ transformer.addReferenceCount();
+ boolean[] selection = new boolean[9];
+ for (int i = 0; i < selection.length; i++) {
+ selection[i] = !partial || i % 2 == 0;
+ }
+ transformer.evaluateWithSelection(selection);
+ Column result = transformer.getColumn();
+ Assert.assertEquals(expected.length, result.getPositionCount());
+ for (int i = 0; i < expected.length; i++) {
+ if (!selection[i] || expected[i] == null) {
+ Assert.assertTrue(result.isNull(i));
+ } else {
+ Assert.assertEquals(expected[i], result.getBoolean(i));
Review Comment:
This branch only checks `getBoolean(i)`, and a NULL position in
`BooleanColumn` reads as `false`, so the test can't tell FALSE from NULL. That
leaves the "left is FALSE, so the result is FALSE" half of the fix unprotected:
- Changing line 73 of `LogicAndColumnTransformer` to `builder.appendNull()`
makes rows 3-5 (left = false) return NULL, and both new tests still pass.
- With the pre-fix code, the test only catches row 2 (`NULL AND TRUE`, line
166). Rows 3-5 also came back NULL there, which the assertion on this line
accepts.
The loop also reads `selection` after `evaluateWithSelection` has run, so it
wouldn't notice the method mutating it: dropping the `selection.clone()` on
line 36 likewise turns rows 3-5 into NULL, and the tests still pass.
Suggestion: keep a copy of the selection and assert non-null explicitly.
```java
boolean[] selected = selection.clone();
transformer.evaluateWithSelection(selection);
Column result = transformer.getColumn();
Assert.assertEquals(expected.length, result.getPositionCount());
for (int i = 0; i < expected.length; i++) {
if (!selected[i] || expected[i] == null) {
Assert.assertTrue(result.isNull(i));
} else {
Assert.assertFalse(result.isNull(i));
Assert.assertEquals(expected[i], result.getBoolean(i));
}
}
```
With these assertions, the pre-fix code and both changes above fail, and the
current code passes.
Minor: the partial mask `i % 2 == 0` leaves rows 1 and 7 unselected, and
they are the only rows that reach the "right is false" branch (lines 58-60), so
that branch never runs with a partial selection. A mask that keeps row 1 or 7
selected would cover it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]