[
https://issues.apache.org/jira/browse/SPARK-13690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-13690.
-------------------------------
Resolution: Not A Problem
OK, I'm going to consider this not a (Spark) problem for the foreseeable future.
> UnsafeShuffleWriterSuite fails on arm64 (SnappyError, no native library is
> found)
> ---------------------------------------------------------------------------------
>
> Key: SPARK-13690
> URL: https://issues.apache.org/jira/browse/SPARK-13690
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.0.0
> Environment: $ java -version
> java version "1.8.0_73"
> Java(TM) SE Runtime Environment (build 1.8.0_73-b02)
> Java HotSpot(TM) 64-Bit Server VM (build 25.73-b02, mixed mode)
> $ uname -a
> Linux spark-on-arm 4.2.0-55598-g45f70e3 #5 SMP Tue Feb 2 10:14:08 CET 2016
> aarch64 aarch64 aarch64 GNU/Linux
> Reporter: Santiago M. Mola
> Priority: Minor
> Labels: arm64, porting
>
> UnsafeShuffleWriterSuite fails because of missing Snappy native library on
> arm64.
> {code}
> Tests run: 19, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 6.437 sec
> <<< FAILURE! - in org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite
> mergeSpillsWithFileStreamAndSnappy(org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite)
> Time elapsed: 0.072 sec <<< ERROR!
> java.lang.reflect.InvocationTargetException
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.testMergingSpills(UnsafeShuffleWriterSuite.java:337)
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.mergeSpillsWithFileStreamAndSnappy(UnsafeShuffleWriterSuite.java:389)
> Caused by: java.lang.IllegalArgumentException: org.xerial.snappy.SnappyError:
> [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Linux
> and os.arch=aarch64
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.testMergingSpills(UnsafeShuffleWriterSuite.java:337)
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.mergeSpillsWithFileStreamAndSnappy(UnsafeShuffleWriterSuite.java:389)
> Caused by: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no
> native library is found for os.name=Linux and os.arch=aarch64
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.testMergingSpills(UnsafeShuffleWriterSuite.java:337)
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.mergeSpillsWithFileStreamAndSnappy(UnsafeShuffleWriterSuite.java:389)
> mergeSpillsWithTransferToAndSnappy(org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite)
> Time elapsed: 0.041 sec <<< ERROR!
> java.lang.reflect.InvocationTargetException
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.testMergingSpills(UnsafeShuffleWriterSuite.java:337)
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.mergeSpillsWithTransferToAndSnappy(UnsafeShuffleWriterSuite.java:384)
> Caused by: java.lang.IllegalArgumentException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.xerial.snappy.Snappy
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.testMergingSpills(UnsafeShuffleWriterSuite.java:337)
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.mergeSpillsWithTransferToAndSnappy(UnsafeShuffleWriterSuite.java:384)
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.xerial.snappy.Snappy
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.testMergingSpills(UnsafeShuffleWriterSuite.java:337)
> at
> org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite.mergeSpillsWithTransferToAndSnappy(UnsafeShuffleWriterSuite.java:384)
> Running org.apache.spark.JavaAPISuite
> Tests run: 90, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.526 sec -
> in org.apache.spark.JavaAPISuite
> Running org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.761 sec -
> in org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
> Running org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.967 sec -
> in org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
> Running org.apache.spark.api.java.OptionalSuite
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec -
> in org.apache.spark.api.java.OptionalSuite
> Results :
> Tests in error:
>
> UnsafeShuffleWriterSuite.mergeSpillsWithFileStreamAndSnappy:389->testMergingSpills:337
> » InvocationTarget
>
> UnsafeShuffleWriterSuite.mergeSpillsWithTransferToAndSnappy:384->testMergingSpills:337
> » InvocationTarget
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
