yashmayya commented on code in PR #14163:
URL: https://github.com/apache/pinot/pull/14163#discussion_r1792979784
##########
pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/NumericalFilterOptimizer.java:
##########
@@ -346,6 +365,156 @@ private static Expression
rewriteRangeExpression(Expression range, FilterKind ki
return range;
}
+ /**
+ * Rewrite expressions of the form "column BETWEEN lower AND upper" to
ensure that lower and upper bounds are the same
+ * datatype as the column (or can be cast to the same datatype in the
server).
+ */
+ private static Expression rewriteBetweenExpression(Expression between,
DataType dataType) {
+ List<Expression> operands = between.getFunctionCall().getOperands();
+ Expression lower = operands.get(1);
+ Expression upper = operands.get(2);
+
+ if (lower.isSetLiteral()) {
+ switch (lower.getLiteral().getSetField()) {
+ case LONG_VALUE: {
+ long actual = lower.getLiteral().getLongValue();
+ // Other data types can be converted on the server side.
Review Comment:
> Do you mean we should not rewrite BETWEEN the same way as other range
filters
Yeah, basically this. For instance, taking `intCol >= 2.5` as an example.
`2.5` is cast to `2` (int), and then the `>=` is rewritten to `>` because
`actual - converted > 0` resulting in `intCol > 2`. For `BETWEEN`, we want to
instead rewrite `intCol BETWEEN 2.5 AND y` to `intCol BETWEEN 3 AND y`. We
could change the logic for regular range filter to rewrite `intCol >= 2.5` to
`intCol >= 3` instead to match the `BETWEEN` rewrite logic - is that what
you're suggesting? There are some other differences too though. For instance,
`floatCol < longLiteral` can be rewritten to `floatCol <= castedFloatLiteral`
depending on the comparison between `longLiteral` and `castedFloatLiteral`. We
can't do the same for `BETWEEN` though, and we simply skip any conversion in
these cases, allowing the server to do the cast. Given these differences, it
seemed better overall to keep these rewrites separate, what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
