voonhous commented on code in PR #8380:
URL: https://github.com/apache/hudi/pull/8380#discussion_r1169659091
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -1040,6 +1035,32 @@ private static Object
rewritePrimaryTypeWithDiffSchemaType(Object oldValue, Sche
throw new AvroRuntimeException(String.format("cannot support rewrite value
for schema type: %s since the old schema type is: %s", newSchema, oldSchema));
}
+ /**
+ * When there is a lost in scale, rounding is required to be performed when
performing casts.
+ * For types that have a fix scale, the results of the cast will need to be
the same for COW and MOR.
+ * <p>
+ * NOTE: COW uses Spark's default rounding of HALF_UP (if Spark entry points
are used).
+ * <p>
+ * Floating point types are not considered here as they are susceptible to
floating point rounding errors.
+ * <p>
+ * Summary:
+ * <ul>
+ * <li>double/string -> decimal: HALF_UP</li>
+ * <li>float -> decimal: HALF_EVEN</li>
+ * </ul>
Review Comment:
My initial considerations for only applying this to `DOUBLE` and `STRING`
was because these are exact representation of fractional numbers.
i.e. 1.5 = 1.5.
For any other types, like float, these are not exact.
i.e. 1.5 is usually represented as 1.49918237
As such, given that `HALF_EVEN` was introduced in #7769, i wanted to limit
the change to the types to exact types.
Since we want to standardize all rounding mode to `HALF_UP`, this method is
no long required then. I will standardise everything to `HALF_UP`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
