Thanks, Sean. I will explore online more. Regards, _________________________________ *Md. Rezaul Karim*, BSc, MSc PhD Researcher, INSIGHT Centre for Data Analytics National University of Ireland, Galway IDA Business Park, Dangan, Galway, Ireland Web: http://www.reza-analytics.eu/index.html <http://139.59.184.114/index.html>
On 19 January 2017 at 10:59, Sean Owen <so...@cloudera.com> wrote: > It's a message from Hadoop libs, not Spark. It can be safely ignored. It's > just saying you haven't installed the additional (non-Apache-licensed) > native libs that can accelerate some operations. This is something you can > easily have read more about online. > > On Thu, Jan 19, 2017 at 10:57 AM Md. Rezaul Karim < > rezaul.ka...@insight-centre.org> wrote: > >> Hi All, >> >> I'm the getting the following WARNING while running Spark jobs in >> standalone mode: >> Unable to load native-hadoop library for your platform... using >> builtin-java classes where applicable >> >> Please note that I have configured the native path and the other ENV >> variables as follows: >> export JAVA_HOME=/usr/lib/jvm/java-8-oracle >> export HADOOP_HOME=/usr/local/hadoop >> export HADOOP_COMMON_LIB_NATIVE_DIR=/usr/local/hadoop/lib/native >> export LD_LIBRARY_PATH=/usr/local/hadoop/lib/native/:$LD_LIBRARY_PATH >> export JAVA_LIBRARY_PATH=/usr/local/hadoop/lib/native >> export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native" >> >> >> Although my Spark job executes successfully and writes the results to a >> file at the end. However, I am not getting any logs to track the progress. >> >> Could someone help me to solve this problem? >> >> >> >> >> Regards, >> _________________________________ >> *Md. Rezaul Karim*, BSc, MSc >> PhD Researcher, INSIGHT Centre for Data Analytics >> National University of Ireland, Galway >> IDA Business Park, Dangan, Galway, Ireland >> Web: http://www.reza-analytics.eu/index.html >> <http://139.59.184.114/index.html> >> >
