JackieTien97 commented on code in PR #18982:
URL: https://github.com/apache/iotdb/pull/18982#discussion_r4347935502
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/AnalyzeVisitor.java:
##########
@@ -1606,16 +1607,25 @@ private Expression analyzeWhereSplitByDevice(
schemaTree,
true,
queryContext);
- return convertConJunctionsToWhereExpression(conJunctions);
+ return convertConJunctionsToWhereExpression(analysis, conJunctions);
}
- private Expression convertConJunctionsToWhereExpression(List<Expression>
conJunctions) {
+ private Expression convertConJunctionsToWhereExpression(
+ Analysis analysis, List<Expression> conJunctions) {
Expression predicate =
PredicateUtils.combineConjuncts(
conJunctions.stream().distinct().collect(Collectors.toList()));
predicate = PredicateUtils.simplifyPredicate(predicate);
predicate = normalizeExpression(predicate);
- return predicate;
+ analyzeExpressionType(analysis, predicate);
+ Expression predicateWithoutNot =
+ PredicateUtils.predicateRemoveNot(predicate, analysis::getType);
Review Comment:
Now that `NOT` is removed after binding, a negated disjunction that contains
a time condition leaves a time-only conjunct in the residual WHERE predicate.
For example, `NOT (time > 2 OR s1 > 5)` becomes `time <= 2 AND s1 <= 5`. The
new De Morgan branch in `extractGlobalTimePredicate` copies `time <= 2` into
the global time filter, but the residual still contains it.
When the plan has a `FullOuterTimeJoin`,
`PredicatePushDown#extractPushDownConjunctsForEachChild` calls
`PredicateUtils#extractPredicateSourceSymbol` for every conjunct. That method
assumes the conjunct references at least one series, so `pathIterator.next()`
throws on the time-only conjunct.
Repro (same setup as my first review):
```sql
SELECT s1, s2 FROM root.rv.d1 WHERE NOT (time > 2 OR s1 > 5)
-- expected: (1, 1, 10) this PR: 305 ... executeStatement failed. null
SELECT s1, s2 FROM root.rv.d1 WHERE NOT (time > 2 OR s1 > 5) ALIGN BY DEVICE
-- expected: (1, root.rv.d1, 1, 10) this PR: 305 ... null
SELECT s1, s2 FROM root.rv.d1 WHERE NOT (time > 2 OR s1 > 5) AND s2 > 5
-- expected: (1, 1, 10) this PR: 305 ... null
SELECT s1, s2 FROM root.rv.d1 WHERE NOT ((time > 2 AND time < 4) OR s1 > 5)
-- expected: (1, 1, 10) this PR: 305 ... null
```
```
java.util.NoSuchElementException
at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1607)
at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1628)
at
...PredicateUtils.extractPredicateSourceSymbol(PredicateUtils.java:487)
at
...PredicatePushDown$Rewriter.extractPushDownConjunctsForEachChild(PredicatePushDown.java:201)
at
...PredicatePushDown$Rewriter.visitFullOuterTimeJoin(PredicatePushDown.java:145)
```
These queries only work today in two cases:
- The query reads a single series. The conjuncts then go through
`visitSeriesScanSource`; for example, `SELECT s1 FROM root.rv.d1 WHERE NOT
(time > 2 OR s1 > 5)` pushes `Time <= 2 & root.rv.d1.s1 <= 5` into the scan.
- The device is aligned.
On master these queries fail with 301 `This predicate contains a not!`, so
this isn't a regression. However, `NOT (time > ... OR ...)` is exactly the
shape that 34c39add targets, and it still fails for multi-series queries.
The underlying bug already exists on master. A time-only conjunct that can't
be extracted as a global time filter crashes the same way there, e.g. `SELECT
s1, s2 FROM root.rv.d1 WHERE time % 2 = 0` or `... WHERE time > 2.5`. So I'd
suggest fixing it in `extractPredicateSourceSymbol`: return `null` when the
conjunct references no series, so it stays in the `FilterNode`. Pushing
time-only conjuncts to every child would also work. Either way, both cases are
covered. Optionally, after the rewrite, you could also drop time-only conjuncts
that are already implied by the global time filter from the residual.
Please also add an IT that reads at least two non-aligned series, e.g.
`SELECT s1, s2 ... WHERE NOT (time > 2 OR s1 > 5)`, with and without `ALIGN BY
DEVICE`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]