Funny, I was just trying to add something to the wiki addressing this.
These instructions are for 2.2, but I imagine that 2.0.5 is probably very similar. If the formatting doesn't come through for whatever reason, I posted the same thing here: http://answers.splunk.com/answers/118174/hunk-reports-an-error-with-apache-hadoop?page=1&focusedAnswerId=122311#122311 This isn't necessarily a big problem - Hadoop will function without native libraries. You may find it easier to ignore the message or disable/redirect JVM warnings. You can disable the error message or redirect it to stderr, but that only moves the error out of your way and doesn't deal with the root problem. The root problem is that the hadoop distribution does not include native libraries. They must be compiled from source. You can build your own distribution that includes native libraries using the following steps: 1) Install developer tools and dependencies: 1a) From repositories: apt-get install gcc g++ make maven cmake zlib zlib1g-dev for RedHat environments, you can probably use a similar yum line: yum install gcc g++ make maven cmake zlib zlib-devel There may be some other dependencies or slightly different package names depending on what you already have installed and what OS you are running. If so, some google-able errors will pop up during the rest of the process. 1b) Protocol Buffers From Source: mkdir /tmp/protobuf cd /tmp/protobuf wget http:// protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz tar -xvzf ./protobuf-2.5.0.tar.gz cd protobuf-2.5.0 ./configure --prefix=/usr make sudo make install cd java mvn install mvn package sudo ldconfig cd /tmp rm -rf protobuf 2) download hadoop source: mkdir /tmp/hadoop-build cd /tmp/hadoop-build wget http:// apache.petsads.us/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz tar -xvzf ./hadoop-2.2.0-src.tar.gz cd hadoop-2.2.0-src 3) Edit the hadoop-auth pom file. vi hadoop-common-project/hadoop-auth/pom.xml add the following dependency: <dependency> <groupId>org.mortbay.jetty</groupId> <artifactId>jetty-util</artifactId> <scope>test</scope> </dependency> You should see an already existing dependency that looks very similar if you search for "org.mortbay.jetty", add this dependency above or below it. 3) Compile it: export Platform=x64 cd /tmp/hadoop-build/hadoop-2.2.0-src mvn clean install -DskipTests cd hadoop-mapreduce-project mvn package -Pdist,native -DskipTests=true -Dtar cd /tmp/hadoop-build/hadoop-2.2.0-src mvn package -Pdist,native -DskipTests=true -Dtar 4) Copy your natively compiled distribution somewhere to be saved: cp /tmp/hadoop-build/hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0.tar.gz /my/distribution/share/hadoop-2.2.0.tar.gz 5) Delete the build files (once you are satisfied that everything is working properly): cd /tmp rm -rf hadoop-build Now any fresh installations based on this build will include native 64 bit libraries. You can set up a new instance of hadoop locally, or you can simply overwrite the files in the $HADOOP-INSTALL/lib/native directory with those in your hadoop-2.2.0.tar.gz file. *From:* xeon Mailinglist [mailto:[email protected]] *Sent:* Wednesday, February 12, 2014 9:28 PM *To:* [email protected] *Subject:* Unable to load native-hadoop library for your platform I am trying to run an example and I get the following error: HadoopMaster-nh:~# /root/Programs/hadoop/bin/hdfs dfs -count /wiki OpenJDK 64-Bit Server VM warning: You have loaded library /root/Programs/hadoop-2.0.5-alpha/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'. 14/02/13 05:24:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable I tried to run execstack -c, but the problem stays the same. Any help? HadoopMaster-nh:~# execstack -c /root/Programs/hadoop-2.0.5-alpha/lib/native/libhadoop.so.1.0.0 HadoopMaster-nh:~# /root/Programs/hadoop/bin/hdfs dfs -count /wiki OpenJDK 64-Bit Server VM warning: You have loaded library /root/Programs/hadoop-2.0.5-alpha/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'. 14/02/13 05:26:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
