Akash3121 commented on code in PR #10403:
URL: https://github.com/apache/paimon/pull/10403#discussion_r4209324957
##########
paimon-hive/paimon-hive-connector-common/src/main/java/org/apache/paimon/hive/SearchArgumentToPredicateConverter.java:
##########
@@ -182,6 +184,33 @@ private Object toLiteral(DataType literalType, Object o) {
if (o instanceof HiveDecimalWritable) {
o = ((HiveDecimalWritable) o).getHiveDecimal().bigDecimalValue();
}
- return convertJavaObject(literalType, o);
+ Object literal = convertJavaObject(literalType, o);
+ // Hive does not cast literals to the column type. A literal that
cannot be represented
+ // exactly in the column type (e.g. 1000 for DECIMAL(5, 2), 1.005 for
DECIMAL(5, 2) or 200
+ // for TINYINT) would be altered here and prune data files wrongly, so
leave this
+ // conjunct to Hive's residual filter instead.
+ if (o != null && !isExact(literalType, o, literal)) {
+ throw new UnsupportedOperationException(
+ "Literal "
+ + o
+ + " cannot be represented exactly in column type "
+ + literalType
+ + ".");
+ }
+ return literal;
+ }
+
+ private static boolean isExact(DataType literalType, Object original,
Object literal) {
Review Comment:
P1 - Include FLOAT in the exact-representation check
Please cover FLOAT literals in `isExact` as well. Hive’s FLOAT SARG
representation is a `Double`, while `convertJavaObject` narrows it with
`floatValue()`, and the current default branch accepts that narrowing
unconditionally. For example, a FLOAT value stored as `0.2f` satisfies Hive’s
`f > 0.2D` because the column value is promoted to double (`0.20000000298… >
0.2`). The pushed predicate becomes `f > 0.2f`, evaluates false, and can prune
the matching file before Hive’s residual filter runs. Please reject the
pushdown unless converting the float back to double equals the original
literal, and add converter plus MR/Tez coverage for this boundary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
