Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "UsingLzoCompression" page has been changed by PaulKist. http://wiki.apache.org/hadoop/UsingLzoCompression?action=diff&rev1=19&rev2=20 -------------------------------------------------- * For 0.21 or 0.22 checkout trunk On Mac: - * On Mac you might want to use a command line like: + * To install the hadoop-gpl-compression library on a mac, it is advisable to use MacPorts. To do so you must do the following: + (Parts of this found on http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ ) + + {{{ + > port fetch lzo2 # If for some reason LZO2 is already installed, please uninstall first before doing this + > port edit lzo2 # A vim editor should open + + // Add the following block of text in the file and save the file. + variant x86_64 description "Build the 64-bit." { + configure.args-delete --build=x86-apple-darwin ABI=standard + configure.cflags-delete -m32 + configure.cxxflags-delete -m32 + + configure.args-append --build=x86_64-apple-darwin ABI=64 + configure.cflags-append -m64 -arch x86_64 + configure.cxxflags-append -m64 -arch x86_64 + } + + ## END ## + + > port install lzo2 +x86_64 + }}} + + This ensures the library is built in 64 bit mode, because java 1.6 is 64 bit only. Also to make sure your lzo library is x64_64 as well, type: {{{ + $ file /usr/local/lib/liblzo2.2.0.0.dylib + /usr/local/lib/liblzo2.2.0.0.dylib: Mach-O 64-bit dynamically linked shared library x86_64 + }}} + + + * On Mac you might want to use a command line, in the hadoop-gpl-compression home directory: {{{ env JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/ \ C_INCLUDE_PATH=/path/to/lzo64/include LIBRARY_PATH=/path/to/lzo64/lib \ CFLAGS="-arch x86_64" ant clean compile-native test tar }}} - This ensures the library is built in 64 bit mode, because java 1.6 is 64 bit only. Also make sure your lzo library is x64_64 as well, type: {{{ - $ file /usr/local/lib/liblzo2.2.0.0.dylib - /usr/local/lib/liblzo2.2.0.0.dylib: Mach-O 64-bit dynamically linked shared library x86_64 + + * Note: If you used macports, /path/to/lzo64 will be replaced by /opt/local (e.g. /opt/local/include and /opt/local/lib ) + * Note: If for some reason you are getting compilation errors, you can add the following to the environment variables: + + {{{ + CLASSPATH=$HADOOP_HOME/hadoop-<version>-core.jar }}} + + * Note: Also during this install, if you are running into permission denied errors, even as ROOT, you can go ahead and change permissions of those files in order for the build to complete + + Once the install has completed, a jar file and lib files have been created in the HADOOP-GPL-HOME/build directory. All these files MUST be copied both into your HADOOP_HOME and HBASE_HOME directories using the following commands from the HADOOP-GPL-HOME directory: + + {{{ + > cp build/hadoop-gpl-compression-0.1.0-dev/hadoop-gpl-compression-0.1.0-dev.jar $HADOOP_HOME/lib/ + > cp build/hadoop-gpl-compression-0.1.0-dev/hadoop-gpl-compression-0.1.0-dev.jar $HBASE_HOME/lib/ + > tar -cBf - -C build/hadoop-gpl-compression-0.1.0-dev/lib/native . | tar -xBvf - -C $HADOOP_HOME/lib/native + > tar -cBf - -C build/hadoop-gpl-compression-0.1.0-dev/lib/native . | tar -xBvf - -C $HBASE_HOME/lib/native + }}} + + + To build lzo2 from source in 64 bit mode: {{{ $ CFLAGS="-arch x86_64" ./configure --build=x86_64-darwin --enable-shared --disable-asm <configure output>
