kasakrisz commented on code in PR #4237:
URL: https://github.com/apache/hive/pull/4237#discussion_r1172371274
##########
ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/rules/TestHivePointLookupOptimizerRule.java:
##########
@@ -348,4 +356,100 @@ public void testRecursionIsNotObstructed() {
condition.toString());
}
+ @Test
+ public void testSameVarcharLiteralDifferentPrecision() {
+
+ final RexBuilder rexBuilder = relBuilder.getRexBuilder();
+ RelDataType stringType30 =
rexBuilder.getTypeFactory().createTypeWithCharsetAndCollation(
+ rexBuilder.getTypeFactory().createSqlType(SqlTypeName.VARCHAR, 30),
+ Charset.forName(ConversionUtil.NATIVE_UTF16_CHARSET_NAME),
SqlCollation.IMPLICIT);
+ RexNode lita30 =
rexBuilder.makeLiteral(RexNodeExprFactory.makeHiveUnicodeString("AAA111"),
stringType30, true);
+ RexNode litb30 =
rexBuilder.makeLiteral(RexNodeExprFactory.makeHiveUnicodeString("BBB222"),
stringType30, true);
+
+ RelDataType stringType14 =
rexBuilder.getTypeFactory().createTypeWithCharsetAndCollation(
+ rexBuilder.getTypeFactory().createSqlType(SqlTypeName.VARCHAR, 14),
+ Charset.forName(ConversionUtil.NATIVE_UTF16_CHARSET_NAME),
SqlCollation.IMPLICIT);
+ RexNode lita14 =
rexBuilder.makeLiteral(RexNodeExprFactory.makeHiveUnicodeString("AAA111"),
stringType14, true);
+ RexNode litb14 =
rexBuilder.makeLiteral(RexNodeExprFactory.makeHiveUnicodeString("BBB222"),
stringType14, true);
+
+ final RelNode basePlan = relBuilder
+ .scan("t")
+ .filter(and(relBuilder,
+ relBuilder.call(SqlStdOperatorTable.IN,
relBuilder.field("f2"), lita30, litb30),
+ relBuilder.call(SqlStdOperatorTable.IN,
relBuilder.field("f2"), lita14, litb14)))
+ .build();
+
+ planner.setRoot(basePlan);
+ RelNode optimizedRelNode = planner.findBestExp();
+
+ HiveFilter filter = (HiveFilter) optimizedRelNode;
+ RexNode condition = filter.getCondition();
+ System.out.println(condition);
+ assertEquals("IN($1, " +
+ "_UTF-16LE'AAA111':VARCHAR(30) CHARACTER SET \"UTF-16LE\",
" +
+ "_UTF-16LE'BBB222':VARCHAR(30) CHARACTER SET
\"UTF-16LE\")",
Review Comment:
Unfortunately in Calcite 1.25 `RexSimplify` returns the input expression so
it can not recognize literals with same values and type but different precision.
I also tested a similar expression with Calcite 1.33:
```
AND(OR(=($0, _UTF-16LE'AAA111'), =($0, _UTF-16LE'BBB222')), OR(=($0,
_UTF-16LE'AAA111'), =($0, _UTF-16LE'BBB222')))
```
and I got
```
SEARCH($0, Sarg[_UTF-16LE'AAA111':VARCHAR(30) CHARACTER SET "UTF-16LE",
_UTF-16LE'BBB222':VARCHAR(30) CHARACTER SET "UTF-16LE"]:VARCHAR(30) CHARACTER
SET "UTF-16LE")
```
In Calcite 1.33 IN expression with constants is no longer represented by
`RexCall` but `SEARCH` so I had to transform the original expression to `OR`s
but the literals has different precision.
This time the expression was simplified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
