XorSum commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2053870360
The manual test steps to avoid native zStandard library not available error: 1. a x86_64 linux machine 2. download and extract [hadoop-3.4.0.tar.gz](https://dlcdn.apache.org/hadoop/common/hadoop-3.4.0/hadoop-3.4.0.tar.gz). We can find native lib in `hadoop-3.4.0/lib/native`. 3. start spark-sql with `extraLibraryPath` conf ``` ./bin/spark-sql \ --master local[*] \ --conf spark.driver.extraLibraryPath=/path/to/hadoop-3.4.0/lib/native/lib/native ``` However, I have no idea how to get and set hadoop native lib during unit test, or how to bundle the hadoop native lib into the spark distribution. Could you give me some advice or tips? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
