This is an automated email from the ASF dual-hosted git repository. srowen pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push: new 871aacc [SPARK-29918][SQL][FOLLOWUP][TEST] Fix arrayOffset in `RecordBinaryComparatorSuite` 871aacc is described below commit 871aacc11a4ffe36b89395171c04ee1e1a2a4dda Author: Xingbo Jiang <xingbo.ji...@databricks.com> AuthorDate: Thu Dec 19 17:01:40 2019 +0800 [SPARK-29918][SQL][FOLLOWUP][TEST] Fix arrayOffset in `RecordBinaryComparatorSuite` ### What changes were proposed in this pull request? As mentioned in https://github.com/apache/spark/pull/26548#pullrequestreview-334345333, some test cases in `RecordBinaryComparatorSuite` use a fixed arrayOffset when writing to long arrays, this could lead to weird stuff including crashing with a SIGSEGV. This PR fix the problem by computing the arrayOffset based on `Platform.LONG_ARRAY_OFFSET`. ### How was this patch tested? Tested locally. Previously, when we try to add `System.gc()` between write into long array and compare by RecordBinaryComparator, there is a chance to hit JVM crash with SIGSEGV like: ``` # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007efc66970bcb, pid=11831, tid=0x00007efc0f9f9700 # # JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10) # Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x5fbbcb] # # Core dump written. Default location: /home/jenkins/workspace/sql/core/core or core.11831 # # An error report file with more information is saved as: # /home/jenkins/workspace/sql/core/hs_err_pid11831.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # ``` After the fix those test cases didn't crash the JVM anymore. Closes #26939 from jiangxb1987/rbc. Authored-by: Xingbo Jiang <xingbo.ji...@databricks.com> Signed-off-by: Wenchen Fan <wenc...@databricks.com> --- .../apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql/core/src/test/java/test/org/apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java b/sql/core/src/test/java/test/org/apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java index 9664638e..c832d07 100644 --- a/sql/core/src/test/java/test/org/apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java +++ b/sql/core/src/test/java/test/org/apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java @@ -323,7 +323,7 @@ public class RecordBinaryComparatorSuite { @Test public void testCompareLongsAsLittleEndian() { - long arrayOffset = 12; + long arrayOffset = Platform.LONG_ARRAY_OFFSET + 4; long[] arr1 = new long[2]; Platform.putLong(arr1, arrayOffset, 0x0100000000000000L); @@ -345,7 +345,7 @@ public class RecordBinaryComparatorSuite { @Test public void testCompareLongsAsUnsigned() { - long arrayOffset = 12; + long arrayOffset = Platform.LONG_ARRAY_OFFSET + 4; long[] arr1 = new long[2]; Platform.putLong(arr1, arrayOffset + 4, 0xa000000000000000L); --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org