sumeetgajjar commented on pull request #32114: URL: https://github.com/apache/spark/pull/32114#issuecomment-817050465
> It turns out "on-disk storage" (`encryptionTest`) under `BlockManagerSuite` fails when "encryption = on" on my mac. But Intellij does not report it as a failure since the jvm simply exited. It also does not run the next set of tests under `BlockManagerSuite` given such scenarios and hence I didn't notice NPE for two of those unrunned tests. > > However, the same encryption test passes on Github. JVM exits while dynamically loading `org.apache.commons.crypto.random.OpenSslCryptoRandom` using `commons-crypto` on my machine. > > Lesson learned: always perform a final sanity test run using `sbt/mvn` before concluding the test passes 😋 I had `[email protected]` installed on my mac (Catalina 10.15.7), however, the corresponding shared libs `libcrypto.1.1.dylib` and `libssl.1.1.dylib` were missing from my `/usr/local/lib` dir. Running the following commands solved the issue and I was able to run the `encryptionTest`. ``` cd /usr/local/Cellar/[email protected]/1.1.1k/lib cp libssl.1.1.dylib libcrypto.1.1.dylib /usr/local/lib cd /usr/local/lib ln -s libcrypto.1.1.dylib libcrypto.dylib ln -s libssl.1.1.dylib libssl.dylib ``` You could check if the `commons-crypto` is using the correct `openssl` shared libs by running the following command ``` java -jar Desktop/commons-crypto-1.1.0/commons-crypto-1.1.0.jar Crypto Apache Commons Crypto 1.1.0 Native code loaded OK: 1.1.0 Native name: Apache Commons Crypto Native built: Aug 28 2020 OpenSSL library loaded OK, version: 0x101010bf OpenSSL library info: OpenSSL 1.1.1k 25 Mar 2021 Random instance created OK: org.apache.commons.crypto.random.OpenSslCryptoRandom@2a84aee7 Cipher AES/CTR/NoPadding instance created OK: org.apache.commons.crypto.cipher.OpenSslCipher@1fb3ebeb Additional OpenSSL_version(n) details: 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG 2: built on: Thu Mar 25 21:01:02 2021 UTC 3: platform: darwin64-x86_64-cc 4: OPENSSLDIR: "/usr/local/etc/[email protected]" 5: ENGINESDIR: "/usr/local/Cellar/[email protected]/1.1.1k/lib/engines-1.1" ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
