> > export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true"
>
> Was there actually a problem if you didn't set this?
>
I have commented this out and restarted mapred and everything still worked ok.
It is probably only needed for the HDFS processes.
> > 4. Symink JNI library
> > cd $HADOOP_HOME/lib/native/Linux-amd64-64 (or Linux-i386-32 on 32-bit
> platform)
> > ln -s /usr/lib/jni/libcephfs_jni.so .
> >
> > I have tried adding -Djava.library.path=/usr/lib/jni to HADOOP_OPTS
> in hadoop-env.sh and exporting LD_LIBRARY_PATH=/usr/lib/jni in hadoop-
> env.sh, but it didn't work for me. I'd love to hear about a more
> elegant method of making Hadoop Ceph JNI aware w/o symlinking.
>
> I've never had to do the symlink trick. IIRC what works for me is to
> set LD_LIBRARY_PATH in hadoop-env.sh, and then restart the daemons. The
> Hadoop start-up scripts should update java.library.path automatically.
>
On my system it already fails at 'hadoop fs -ls /' which doesn't require any
Hadoop daemons running.
hadoop script nulls JAVA_LIBRARY_PATH, finds the platform and populates
JAVA_LIBRARY_PATH with ${HADOOP_HOME}/lib/native/${JAVA_PLATFORM} and then sets
-Djava.library.path=$JAVA_LIBRARY_PATH" to the options.
It would prepend this with ${HADOOP_HOME}/build/native/${JAVA_PLATFORM}/lib if
it is present, so if you were building on the box, chances are it'll work.
I have also tried to add /usr/lib/jni to /etc/ld.so.conf.d/ceph.conf and
ldconfig -v lists the libraries correctly, but it didn't help :(
So, with my mix of versions and Hadoop installed from the tarball onto user's
home directory (other poster has mentioned that .deb Hadoop worked for him),
there seem to be 3 ways to make this problem go away:
1. Symlink the library into directory already on the java.library.path
2. Add "JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:/usr/lib/jni" to hadoop script
directly
3. Make directory ${HADOOP_HOME}/build/native/${JAVA_PLATFORM} and symlink
/usr/lib/jni to ${HADOOP_HOME}/build/native/${JAVA_PLATFORM}/lib
Cheers,
Ilja
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed. If
you are not the intended recipient, please note that any review, dissemination,
disclosure, alteration, printing, circulation, retention or transmission of
this e-mail and/or any file or attachment transmitted with it, is prohibited
and may be unlawful. If you have received this e-mail or any file or attachment
transmitted with it in error please notify [email protected]. Although
Openet has taken reasonable precautions to ensure no viruses are present in
this email, we cannot accept responsibility for any loss or damage arising from
the use of this email or attachments.
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
