voonhous commented on PR #8380: URL: https://github.com/apache/hudi/pull/8380#issuecomment-1502754140
> It seems that Spark always uses the HALF_UP rounding mode, the HALF_EVEN mode is only used in tests. So the main reason we use HALF_EVEN for float -> decimal conversion is to avoid error throwing right? HALF_EVEN is used OUTSIDE of tests. For example: https://github.com/apache/hudi/blob/master/hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieInternalRowUtils.scala#L289 As to why HALF_EVEN was used, I am not sure why... But if I were to hazard a guess, it is to avoid bias, where numbers > 0.5 will always be rounded up if HALF_UP was used, where it would shift the average value fo the set of numerics. If HALF_EVEN was used, and there is an equal number of numerics that are > 0.5 that are rounded to the nearest even number in both the UP and DOWN direction, the original average value will not shift as much. Not really sure... Just throwing my assumptions out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
