JackieTien97 commented on code in PR #18982:
URL: https://github.com/apache/iotdb/pull/18982#discussion_r4347935502


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/AnalyzeVisitor.java:
##########
@@ -1606,16 +1607,25 @@ private Expression analyzeWhereSplitByDevice(
             schemaTree,
             true,
             queryContext);
-    return convertConJunctionsToWhereExpression(conJunctions);
+    return convertConJunctionsToWhereExpression(analysis, conJunctions);
   }
 
-  private Expression convertConJunctionsToWhereExpression(List<Expression> 
conJunctions) {
+  private Expression convertConJunctionsToWhereExpression(
+      Analysis analysis, List<Expression> conJunctions) {
     Expression predicate =
         PredicateUtils.combineConjuncts(
             conJunctions.stream().distinct().collect(Collectors.toList()));
     predicate = PredicateUtils.simplifyPredicate(predicate);
     predicate = normalizeExpression(predicate);
-    return predicate;
+    analyzeExpressionType(analysis, predicate);
+    Expression predicateWithoutNot =
+        PredicateUtils.predicateRemoveNot(predicate, analysis::getType);

Review Comment:
   Now that `NOT` is removed after binding, a negated disjunction that contains 
a time condition leaves a time-only conjunct in the residual WHERE predicate. 
For example, `NOT (time > 2 OR s1 > 5)` becomes `time <= 2 AND s1 <= 5`. The 
new De Morgan branch in `extractGlobalTimePredicate` copies `time <= 2` into 
the global time filter, but the residual still contains it.
   
   When the plan has a `FullOuterTimeJoin`, 
`PredicatePushDown#extractPushDownConjunctsForEachChild` calls 
`PredicateUtils#extractPredicateSourceSymbol` for every conjunct. That method 
assumes the conjunct references at least one series, so `pathIterator.next()` 
throws on the time-only conjunct.
   
   Repro (same setup as my first review):
   
   ```sql
   SELECT s1, s2 FROM root.rv.d1 WHERE NOT (time > 2 OR s1 > 5)
   --   expected: (1, 1, 10)    this PR: 305 ... executeStatement failed. null
   SELECT s1, s2 FROM root.rv.d1 WHERE NOT (time > 2 OR s1 > 5) ALIGN BY DEVICE
   --   expected: (1, root.rv.d1, 1, 10)    this PR: 305 ... null
   SELECT s1, s2 FROM root.rv.d1 WHERE NOT (time > 2 OR s1 > 5) AND s2 > 5
   --   expected: (1, 1, 10)    this PR: 305 ... null
   SELECT s1, s2 FROM root.rv.d1 WHERE NOT ((time > 2 AND time < 4) OR s1 > 5)
   --   expected: (1, 1, 10)    this PR: 305 ... null
   ```
   
   ```
   java.util.NoSuchElementException
        at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1607)
        at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1628)
        at 
...PredicateUtils.extractPredicateSourceSymbol(PredicateUtils.java:487)
        at 
...PredicatePushDown$Rewriter.extractPushDownConjunctsForEachChild(PredicatePushDown.java:201)
        at 
...PredicatePushDown$Rewriter.visitFullOuterTimeJoin(PredicatePushDown.java:145)
   ```
   
   These queries only work today in two cases:
   
   - The query reads a single series. The conjuncts then go through 
`visitSeriesScanSource`; for example, `SELECT s1 FROM root.rv.d1 WHERE NOT 
(time > 2 OR s1 > 5)` pushes `Time <= 2 & root.rv.d1.s1 <= 5` into the scan.
   - The device is aligned.
   
   On master these queries fail with 301 `This predicate contains a not!`, so 
this isn't a regression. However, `NOT (time > ... OR ...)` is exactly the 
shape that 34c39add targets, and it still fails for multi-series queries.
   
   The underlying bug already exists on master. A time-only conjunct that can't 
be extracted as a global time filter crashes the same way there, e.g. `SELECT 
s1, s2 FROM root.rv.d1 WHERE time % 2 = 0` or `... WHERE time > 2.5`. So I'd 
suggest fixing it in `extractPredicateSourceSymbol`: return `null` when the 
conjunct references no series, so it stays in the `FilterNode`. Pushing 
time-only conjuncts to every child would also work. Either way, both cases are 
covered. Optionally, after the rewrite, you could also drop time-only conjuncts 
that are already implied by the global time filter from the residual.
   
   Please also add an IT that reads at least two non-aligned series, e.g. 
`SELECT s1, s2 ... WHERE NOT (time > 2 OR s1 > 5)`, with and without `ALIGN BY 
DEVICE`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to