[
https://issues.apache.org/jira/browse/HDFS-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417981#comment-16417981
]
Jason Lowe commented on HDFS-13362:
-----------------------------------
The maven plugin for cmake automatically detects the number of processors in
the machine and uses that for the build parallelism. From CompileMojo#runMake:
{code}
public void runMake() throws MojoExecutionException {
List<String> cmd = new LinkedList<String>();
cmd.add("make");
cmd.add("-j");
cmd.add(String.valueOf(availableProcessors));
cmd.add("VERBOSE=1");
{code}
I'm not an expert on the maven cmake plugin, but I do know the hadoop-common
native builds and hadoop-yarn-server-nodemanager native builds use it. See the
cmake-compile goal definitions in their respective pom files for examples of
how to build with cmake and run cetest for unit tests. Fixing the dependencies
for parallel builds will be a prerequisite since the maven cmake plugin always
builds in parallel.
As for avoiding the build via a maven-level flag rather than a cmake flag, we
should be able to leverage the {{activation}} portion of the profile
configuration in the pom to disable the native build without invoking cmake at
all. HADOOP-13999 did something very similar for the skipShade flag to avoid
the expensive shaded hadoop-client build.
> add a flag to skip the libhdfs++ build
> --------------------------------------
>
> Key: HDFS-13362
> URL: https://issues.apache.org/jira/browse/HDFS-13362
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: James Clampffer
> Priority: Minor
> Attachments: HDFS-13362.000.patch
>
>
> libhdfs++ has significantly increased clean build times for the native client
> on trunk. This covers adding a flag that would let people build libhdfs
> without all of libhdfs++ if they don't need it; it should be built by default
> to maintain compatibility with as many environments as possible.
> Some thoughts:
> -The increase in compile time only impacts clean builds. Incremental
> rebuilds aren't significantly more expensive than they used to be if the code
> hasn't changed.
> -Compile times for libhdfs++ can most likely be reduced but that's a longer
> term project. boost::asio and tr1::optional are header-only libraries that
> are heavily templated so every compilation unit that includes them has to do
> a lot of parsing.
> Is it common to do completely clean builds frequently for interactive users?
> Are there opinions on what would be an acceptable compilation time?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
