Since there might be concerns about the performance degration that might caused by switching to use strictMath, here is some simple performance info: https://bugs.openjdk.java.net/browse/JDK-8210416 , we can see that the perf difference between Math and StrictMath is actually very very small. And there are also some discussions about it in the spark PR: https://github.com/apache/spark/pull/25279 So it shouldn't be a problem.
On Mon, Apr 27, 2020 at 10:46 AM Zhenyu Zheng <[email protected]> wrote: > Hi Hive, > > We have tested and put up a patch for this: > https://issues.apache.org/jira/browse/HIVE-23133 it is done pretty much > the same as in Spark: > https://github.com/apache/spark/pull/25279 , it could be a very good fix > for running Hive across different platforms. > > BR, > > On Fri, Apr 3, 2020 at 3:18 PM Zhenyu Zheng <[email protected]> > wrote: > >> Hi Hive, >> >> Currently, we have set up an ARM CI to test out how Hive works on ARM >> platform: >> >> https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/ >> >> Among the failures, we have observed that some numeric operations can >> have different result across hardware archs, such as: >> >> >> https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/ >> >> >> https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/ >> >> >> https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/ >> >> we can see that the calculation results of log, exp, cos, toRadians etc >> is slitly different than the .out file results that we are >> >> comparing(they are tested and wrote on X86 machines), this is because of >> we use Math Library >> <https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html> for >> these kind of calculations. >> >> and according to the illustration >> <https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html>: >> >> *Unlike some of the numeric methods of class StrictMath, all >> implementations of the equivalent functions of class Math are not* >> *defined to return the bit-for-bit same results. This relaxation permits >> better-performing implementations where strict reproducibility* >> *is not required.* >> >> *By default many of the Math methods simply call the equivalent method in >> StrictMath for their implementation.* >> *Code generators are encouraged to use platform-specific native libraries >> or microprocessor instructions, where available,* >> *to provide higher-performance implementations of Math methods.* >> >> so the result will have difference across hardware archs. >> >> On the other hand, JAVA provided another library StrictMath >> <https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html> >> that will not have this kind of problem as according to its' reference >> <https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html>: >> >> *To help ensure portability of Java programs, the definitions of some of >> the numeric functions in this package require that they produce* >> *the same results as certain published algorithms.* >> >> So in order to fix the above mentioned problem, we have to consider >> switch to use StrictMath instead of Math. >> >> I have also reported an issue about this >> https://issues.apache.org/jira/browse/HIVE-23133 so comments and >> suggestions are welcome. >> >> >> BR, >> >> Zhenyu Zheng >> >> >> >
