Akash3121 commented on code in PR #10403:
URL: https://github.com/apache/paimon/pull/10403#discussion_r4215210865
##########
paimon-hive/paimon-hive-connector-common/src/main/java/org/apache/paimon/hive/SearchArgumentToPredicateConverter.java:
##########
@@ -182,6 +184,33 @@ private Object toLiteral(DataType literalType, Object o) {
if (o instanceof HiveDecimalWritable) {
o = ((HiveDecimalWritable) o).getHiveDecimal().bigDecimalValue();
}
- return convertJavaObject(literalType, o);
+ Object literal = convertJavaObject(literalType, o);
+ // Hive does not cast literals to the column type. A literal that
cannot be represented
+ // exactly in the column type (e.g. 1000 for DECIMAL(5, 2), 1.005 for
DECIMAL(5, 2) or 200
+ // for TINYINT) would be altered here and prune data files wrongly, so
leave this
+ // conjunct to Hive's residual filter instead.
+ if (o != null && !isExact(literalType, o, literal)) {
+ throw new UnsupportedOperationException(
+ "Literal "
+ + o
+ + " cannot be represented exactly in column type "
+ + literalType
+ + ".");
+ }
+ return literal;
+ }
+
+ private static boolean isExact(DataType literalType, Object original,
Object literal) {
Review Comment:
Thanks for checking this so thoroughly, you’re right. I missed the earlier
constant narrowing in `TypeCheckProcFactory`. Hive replaces the comparison
constant with a `Float` before residual evaluation and SARG construction;
`ConvertAstToSearchArg` then only widens that already-rounded float to the
SARG’s Double container, so Paimon’s `floatValue()` recovers the same value.
My `0.2D` example incorrectly assumed Hive retained the original double
semantics.
Your MR/Tez coverage across the prunable and non-prunable layouts also
confirms there is no divergence for the compound operators. I don’t think a
defensive FLOAT branch is necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
