Dongjoon Hyun created SPARK-49055:
-------------------------------------
Summary: Fix OrcEncryptionSuite failure on Mac environment
Key: SPARK-49055
URL: https://issues.apache.org/jira/browse/SPARK-49055
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.0.0
Reporter: Dongjoon Hyun
{code}
git reset --hard 49b4c3bc9c09325de941dfaf41e4fd3a4a4c345f //
[SPARK-45393][BUILD] Upgrade Hadoop to 3.4.0
build/sbt clean "sql/testOnly
org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite"
...
[info] OrcEncryptionSuite:
12:42:55.441 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
12:42:57.950 WARN org.apache.hadoop.crypto.OpensslCipher: Failed to load
OpenSSL Cipher.
java.lang.UnsatisfiedLinkError: 'boolean
org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl()'
at org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl(Native
Method)
at
org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:86)
at
org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec.<init>(OpensslAesCtrCryptoCodec.java:36)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
[info] - Write and read an encrypted file (2 seconds, 486 milliseconds)
[info] - Write and read an encrypted table (402 milliseconds)
[info] - SPARK-35325: Write and read encrypted nested columns (299 milliseconds)
[info] - SPARK-35992: Write and read fully-encrypted columns with default
masking (623 milliseconds)
12:42:59.856 WARN
org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite:
===== POSSIBLE THREAD LEAK IN SUITE
o.a.s.sql.execution.datasources.orc.OrcEncryptionSuite, threads: rpc-boss-3-1
(daemon=true), Thread-17 (daemon=true), ForkJoinPool.commonPool-worker-2
(daemon=true), shuffle-boss-6-1 (daemon=true), ForkJoinPool.commonPool-worker-1
(daemon=true), Thread-18 (daemon=true), ForkJoinPool.commonPool-worker-3
(daemon=true) =====
[info] Run completed in 5 seconds, 291 milliseconds.
[info] Total number of tests run: 4
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
