Hi, I'm using spark-ec2 scripts, and spark applications do not load native hadoop libraries. I've set the native lib path like this:
export SPARK_LIBRARY_PATH='/root/ephemeral-hdfs/lib/native/' But get these warnings in log: WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable WARN LoadSnappy: Snappy native library not loaded Is SPARK_LIBRARY_PATH the right variable for this? Does spark use this variable, or does my application have to set up the native libraries?
