[
https://issues.apache.org/jira/browse/HADOOP-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David James updated HADOOP-14064:
---------------------------------
Description:
This bug prevents me from building a native Hadoop for my system.
The current build fails for me if zlib is not located the usual location(s).
I looked into ways to pass the necessary information to the Maven build. I
found nothing that worked short of modifying the pom.xml of hadoop-common.
Luckily, this is easy for the project maintainers to fix -- see patch below --
since the build already includes the FindZLIB module for cmake
(https://cmake.org/cmake/help/v3.0/module/FindZLIB.html). All that is missing
is to allow maven command line parameters pass through to FindZLIB.
I suggest the following patch (works for me) that accepts two command line
arguments, `zlib.lib` and `zlib.include` and passes them to cmake as
ZLIB_LIBRARY and ZLIB_INCLUDE_DIR, respectively.
It would be used like this:
{code:none}
mvn package -Pdist,native -DskipTests -Dtar \
-Dzlib.lib=$HOME/lib/libz.so \
-Dzlib.include=$HOME/include
{code}
Here is my suggested patch:
{code:none}
diff -r hadoop-2.7.3-src/hadoop-common-project/hadoop-common/pom.xml
hadoop-2.7.3-src-patched/hadoop-common-project/hadoop-common/pom.xml
521a522,523
> <zlib.lib></zlib.lib>
> <zlib.include></zlib.include>
605c607
< <arg line="${basedir}/src/
-DGENERATED_JAVAH=${project.build.directory}/native/javah
-DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DREQUIRE_BZIP2=${require.bzip2}
-DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix}
-DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include}
-DREQUIRE_OPENSSL=${require.openssl} -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix}
-DCUSTOM_OPENSSL_LIB=${openssl.lib} -DCUSTOM_OPENSSL_INCLUDE=${openssl.include}
-DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
---
> <arg line="${basedir}/src/
> -DGENERATED_JAVAH=${project.build.directory}/native/javah
> -DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DZLIB_LIBRARY=${zlib.lib}
> -DZLIB_INCLUDE_DIR=${zlib.include} -DREQUIRE_BZIP2=${require.bzip2}
> -DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix}
> -DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include}
> -DREQUIRE_OPENSSL=${require.openssl}
> -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix} -DCUSTOM_OPENSSL_LIB=${openssl.lib}
> -DCUSTOM_OPENSSL_INCLUDE=${openssl.include}
> -DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
643a646,647
> <zlib.lib></zlib.lib>
> <zlib.include></zlib.include>
{code}
Suggestions and improvements are welcome.
was:
This bug prevents me from building a native Hadoop for my system.
The current build fails for me if zlib is not located the usual location(s).
I looked into ways to pass the necessary information to the Maven build. I
found nothing that worked short of modifying the pom.xml of hadoop-common.
Luckily, this is easy for the project maintainers to fix -- see patch below --
since the build already includes the FindZLIB module for cmake
(https://cmake.org/cmake/help/v3.0/module/FindZLIB.html). All that is missing
is to allow maven command line parameters pass through to FindZLIB.
I suggest the following patch (works for me) that accepts two command line
arguments, `zlib.lib` and `zlib.include` and passes them to cmake as
ZLIB_LIBRARY and ZLIB_INCLUDE_DIR, respectively.
It would be used like this:
{code:none}
mvn package -Pdist,native -DskipTests -Dtar \
-Dzlib.lib=$HOME/lib/libz.so \
-Dzlib.include=$HOME/include
{code}
; e.g. "-Dzlib.lib=$HOME/lib/libz.dylib -Dzlib.include=$HOME/include".
{code:none}
diff -r hadoop-2.7.3-src/hadoop-common-project/hadoop-common/pom.xml
hadoop-2.7.3-src-patched/hadoop-common-project/hadoop-common/pom.xml
521a522,523
> <zlib.lib></zlib.lib>
> <zlib.include></zlib.include>
605c607
< <arg line="${basedir}/src/
-DGENERATED_JAVAH=${project.build.directory}/native/javah
-DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DREQUIRE_BZIP2=${require.bzip2}
-DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix}
-DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include}
-DREQUIRE_OPENSSL=${require.openssl} -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix}
-DCUSTOM_OPENSSL_LIB=${openssl.lib} -DCUSTOM_OPENSSL_INCLUDE=${openssl.include}
-DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
---
> <arg line="${basedir}/src/
> -DGENERATED_JAVAH=${project.build.directory}/native/javah
> -DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DZLIB_LIBRARY=${zlib.lib}
> -DZLIB_INCLUDE_DIR=${zlib.include} -DREQUIRE_BZIP2=${require.bzip2}
> -DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix}
> -DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include}
> -DREQUIRE_OPENSSL=${require.openssl}
> -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix} -DCUSTOM_OPENSSL_LIB=${openssl.lib}
> -DCUSTOM_OPENSSL_INCLUDE=${openssl.include}
> -DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
643a646,647
> <zlib.lib></zlib.lib>
> <zlib.include></zlib.include>
{code}
Suggestions and improvements are welcome.
> Native maven build fails for non-default zlib location
> ------------------------------------------------------
>
> Key: HADOOP-14064
> URL: https://issues.apache.org/jira/browse/HADOOP-14064
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, native
> Affects Versions: 2.7.3
> Reporter: David James
> Labels: build-problem, native
> Fix For: 2.7.3
>
>
> This bug prevents me from building a native Hadoop for my system.
> The current build fails for me if zlib is not located the usual location(s).
> I looked into ways to pass the necessary information to the Maven build. I
> found nothing that worked short of modifying the pom.xml of hadoop-common.
> Luckily, this is easy for the project maintainers to fix -- see patch below
> -- since the build already includes the FindZLIB module for cmake
> (https://cmake.org/cmake/help/v3.0/module/FindZLIB.html). All that is missing
> is to allow maven command line parameters pass through to FindZLIB.
> I suggest the following patch (works for me) that accepts two command line
> arguments, `zlib.lib` and `zlib.include` and passes them to cmake as
> ZLIB_LIBRARY and ZLIB_INCLUDE_DIR, respectively.
> It would be used like this:
> {code:none}
> mvn package -Pdist,native -DskipTests -Dtar \
> -Dzlib.lib=$HOME/lib/libz.so \
> -Dzlib.include=$HOME/include
> {code}
> Here is my suggested patch:
> {code:none}
> diff -r hadoop-2.7.3-src/hadoop-common-project/hadoop-common/pom.xml
> hadoop-2.7.3-src-patched/hadoop-common-project/hadoop-common/pom.xml
> 521a522,523
> > <zlib.lib></zlib.lib>
> > <zlib.include></zlib.include>
> 605c607
> < <arg line="${basedir}/src/
> -DGENERATED_JAVAH=${project.build.directory}/native/javah
> -DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DREQUIRE_BZIP2=${require.bzip2}
> -DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix}
> -DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include}
> -DREQUIRE_OPENSSL=${require.openssl}
> -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix} -DCUSTOM_OPENSSL_LIB=${openssl.lib}
> -DCUSTOM_OPENSSL_INCLUDE=${openssl.include}
> -DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
> ---
> > <arg line="${basedir}/src/
> > -DGENERATED_JAVAH=${project.build.directory}/native/javah
> > -DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DZLIB_LIBRARY=${zlib.lib}
> > -DZLIB_INCLUDE_DIR=${zlib.include} -DREQUIRE_BZIP2=${require.bzip2}
> > -DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix}
> > -DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include}
> > -DREQUIRE_OPENSSL=${require.openssl}
> > -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix}
> > -DCUSTOM_OPENSSL_LIB=${openssl.lib}
> > -DCUSTOM_OPENSSL_INCLUDE=${openssl.include}
> > -DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
> 643a646,647
> > <zlib.lib></zlib.lib>
> > <zlib.include></zlib.include>
> {code}
> Suggestions and improvements are welcome.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
