How to recompiling library?Cause my hadoop was built from tar ball and in /lib/native there exist files like *.a and *.so.Does I need build from source code to recompiling library?
On Tue, Mar 24, 2015 at 6:35 PM, Tsuyoshi Ozawa <[email protected]> wrote: > Sometimes compiled native libraries included in tar ball doesn't work > correctly - how about recompiling library on your environment? > > Thanks, > - Tsuyoshi > > On Tue, Mar 24, 2015 at 6:18 PM, 王鹏飞 <[email protected]> wrote: > > I noticed a map-reduce job encountered an > > > Exception:java.lang.UnsatisfiedLinkError:org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z > > I googled it and released that it was lack of snappy tool.I > > Installed the snappy and copied libsnappy.so* to $HADOOP_HOME/lib/native > > In .bashrc added: > > export LD_LIBRARY_PATH="$HADOOP_HOME/lib/native/" > > export JAVA_LIBRARY_PATH="$HADOOP_HOME/lib/native/" > > export > > HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native" > > In core-site.xml added: > > <property> > > <name>io.compression.codecs</name> > > <value> > > > > org.apache.hadoop.io.compress.GzipCodec, > > > > org.apache.hadoop.io.compress.DefaultCodec, > > > > org.apache.hadoop.io.compress.BZip2Codec, > > > > org.apache.hadoop.io.compress.SnappyCodec > > </value> > > </property> > > In mapred-site.xml added: > > <property> > > > <name>mapreduce.map.output.compress</name> > > <value>true</value> > > </property> > > > > <property> > > > <name>mapred.map.output.compress.codec</name> > > > > <value>org.apache.hadoop.io.compress.SnappyCodec</value> > > </property> > > > > <property> > > <name>mapreduce.admin.user.env</name> > > > > > <value>LD_LIBRARY_PATH=/home/hadoop/software/hadoop-2.4.1/lib/native</value> > > </property> > > In yarn-site.xml added: > > <property> > > <name>yarn.app.mapreduce.am.env</name> > > > > > <value>LD_LIBRARY_PATH=/home/hadoop/software/hadoop-2.4.1/lib/native</value> > > </property> > > Finally make the same setting in the datanode ,but it still > didn't > > work 。But in a Pseudo-Distributed cluster, I only copy the libsnappy.so* > to > > $HADOOP_HOME/lib/native without changing the configuration ,it works well > > .Why in this cluster it appears so difficult? > > > > > > > > > > > > >
