LuciferYang commented on PR #45583:
URL: https://github.com/apache/spark/pull/45583#issuecomment-2257456279
@dongjoon-hyun Test on master:
before this commit:
```
git reset --hard a34c8ceb19bd1c1548a60bb144d1c587a2861cd8 //
[SPARK-47462][SQL] Align mappings of other unsigned numeric types with TINYINT
in MySQLDialect
build/sbt clean "sql/testOnly
org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite"
```
result:
```
[info] OrcEncryptionSuite:
12:47:33.148 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
[info] - Write and read an encrypted file (2 seconds, 117 milliseconds)
[info] - Write and read an encrypted table (375 milliseconds)
[info] - SPARK-35325: Write and read encrypted nested columns (275
milliseconds)
[info] - SPARK-35992: Write and read fully-encrypted columns with default
masking (517 milliseconds)
12:47:37.058 WARN
org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite:
===== POSSIBLE THREAD LEAK IN SUITE
o.a.s.sql.execution.datasources.orc.OrcEncryptionSuite, threads:
ForkJoinPool.commonPool-worker-4 (daemon=true), rpc-boss-3-1 (daemon=true),
Thread-17 (daemon=true), ForkJoinPool.commonPool-worker-2 (daemon=true),
shuffle-boss-6-1 (daemon=true), ForkJoinPool.commonPool-worker-1 (daemon=true),
Thread-18 (daemon=true), ForkJoinPool.commonPool-worker-3 (daemon=true) =====
[info] Run completed in 4 seconds, 760 milliseconds.
[info] Total number of tests run: 4
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
```
after this commit:
```
git reset --hard 49b4c3bc9c09325de941dfaf41e4fd3a4a4c345f //
[SPARK-45393][BUILD] Upgrade Hadoop to 3.4.0
build/sbt clean "sql/testOnly
org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite"
```
result:
```
[info] OrcEncryptionSuite:
12:42:55.441 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
12:42:57.950 WARN org.apache.hadoop.crypto.OpensslCipher: Failed to load
OpenSSL Cipher.
java.lang.UnsatisfiedLinkError: 'boolean
org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl()'
at org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl(Native
Method)
at
org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:86)
at
org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec.<init>(OpensslAesCtrCryptoCodec.java:36)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
[info] - Write and read an encrypted file (2 seconds, 486 milliseconds)
[info] - Write and read an encrypted table (402 milliseconds)
[info] - SPARK-35325: Write and read encrypted nested columns (299
milliseconds)
[info] - SPARK-35992: Write and read fully-encrypted columns with default
masking (623 milliseconds)
12:42:59.856 WARN
org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite:
===== POSSIBLE THREAD LEAK IN SUITE
o.a.s.sql.execution.datasources.orc.OrcEncryptionSuite, threads: rpc-boss-3-1
(daemon=true), Thread-17 (daemon=true), ForkJoinPool.commonPool-worker-2
(daemon=true), shuffle-boss-6-1 (daemon=true), ForkJoinPool.commonPool-worker-1
(daemon=true), Thread-18 (daemon=true), ForkJoinPool.commonPool-worker-3
(daemon=true) =====
[info] Run completed in 5 seconds, 291 milliseconds.
[info] Total number of tests run: 4
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
