soumyakanti3578 commented on code in PR #5196:
URL: https://github.com/apache/hive/pull/5196#discussion_r1673136834
##########
ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/stats/TestFilterSelectivityEstimator.java:
##########
@@ -508,8 +492,19 @@ public void
testComputeRangePredicateSelectivityBetweenWithNULLS() {
public void testComputeRangePredicateSelectivityNotBetweenWithNULLS() {
doReturn((double) 20).when(tableMock).getRowCount();
doReturn(Collections.singletonList(stats)).when(tableMock).getColStat(Collections.singletonList(0));
- RexNode filter = REX_BUILDER.makeCall(SqlStdOperatorTable.BETWEEN,
boolTrue, inputRef0, int1, int3);
+ RexNode filter = makeNotBetween(inputRef0, int1, int3);
FilterSelectivityEstimator estimator = new
FilterSelectivityEstimator(scan, mq);
Assert.assertEquals(0.55, estimator.estimateSelectivity(filter), DELTA);
}
+
+ private RexNode makeNotBetween(RexNode inputRef, RexNode left, RexNode
right) {
+ RexNode withOr = REX_BUILDER.makeCall(
+ SqlStdOperatorTable.OR,
+ REX_BUILDER.makeCall(SqlStdOperatorTable.LESS_THAN, inputRef, left),
+ REX_BUILDER.makeCall(SqlStdOperatorTable.GREATER_THAN, inputRef, right)
+ );
+ RexSimplify simplify = new RexSimplify(REX_BUILDER,
RelOptPredicateList.EMPTY, RexUtil.EXECUTOR);
+
+ return simplify.simplify(withOr);
Review Comment:
Unfortunately, for NOT, we instead call the method
`computeNotEqualitySelectivity` and it returns incorrect results, as described
above. Maybe we need to check if NOT is implemented correctly, however, I am
not very knowledgeable with the theory of selectivity calculations.
I have noticed through tests that NOT BETWEEN never adds a NOT in front of
the SEARCH operator, like you have described above. It instead includes
multiple ranges in the Sarg. However, we should definitely ensure that we
handle NOT(SEARCH(...)) correctly in Hive as it is possible make such a
RexNode. I will try to look into this, however, if you have some time, could
you also look into this please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
