[
https://issues.apache.org/jira/browse/HADOOP-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106978#comment-15106978
]
Alan Burlison commented on HADOOP-11505:
----------------------------------------
You can't buildhadoop-mapreduce-project or hadoop-mapreduce-client on their own
anyway, despite what it says in BUILDING.txt - or at least I can't:
{noformat}
Where to run Maven from?
It can be run from any module. The only catch is that if not run from utrunk
all modules that are not part of the build run must be installed in the local
Maven cache or available in a Maven repository.
{noformat}
If I try to build from either {{hadoop-mapreduce-project}} or
{{hadoop-mapreduce-project/hadoop-mapreduce-client}} it results in:
{noformat}
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR]
/pool/home/alanbur/bigdata/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/TaskAttemptFinishingMonitor.java:[41,42]
SystemClock() has private access in org.apache.hadoop.yarn.util.SystemClock
[ERROR]
/pool/home/alanbur/bigdata/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java:[248,9]
SystemClock() has private access in org.apache.hadoop.yarn.util.SystemClock
[INFO] 2 errors
{noformat}
The only way I can get it to build is to do a full build or to use {{mvn -pl
hadoop-mapreduce-project -am compile}} from the repo root directory. And if I
do that it only takes 33 seconds to do a clean build versus 19 seconds to do a
nothing-has-changed rebuild. That's because stuff gets needlessly rebuilt even
when nothing has changed:
{noformat}
[INFO] Compiling 862 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-common-project/hadoop-common/target/classes
[INFO] Compiling 228 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/target/classes
{noformat}
I note also that the above incantation already builds Hadoop-Common, so there's
no increase in build time anyway. If there's a problem to be fixed it's the one
that causes 3180 files to be unnecessarily recompiled during a full build even
when nothing has changed:
{noformat}
[INFO] Compiling 862 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-common-project/hadoop-common/target/classes
[INFO] Compiling 228 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/target/classes
[INFO] Compiling 229 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/target/classes
[INFO] Compiling 346 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes
[INFO] Compiling 78 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/classes
[INFO] Compiling 217 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
[INFO] Compiling 388 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/classes
[INFO] Compiling 48 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/target/classes
[INFO] Compiling 507 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/target/classes
[INFO] Compiling 129 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/target/classes
[INFO] Compiling 48 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/target/classes
[INFO] Compiling 95 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-tools/hadoop-rumen/target/classes
[INFO] Compiling 5 source files to
/pool/home/alanbur/bigdata/hadoop/hadoop-tools/hadoop-extras/target/classes
{noformat}
And that's just the Java files. If the unnecessary regeneration of Makefiles
and the consequential recompilations were prevented then even more time would
be saved.
> Various native parts use bswap incorrectly and unportably
> ---------------------------------------------------------
>
> Key: HADOOP-11505
> URL: https://issues.apache.org/jira/browse/HADOOP-11505
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Colin Patrick McCabe
> Assignee: Alan Burlison
> Fix For: 3.0.0
>
> Attachments: HADOOP-11505.001.patch, HADOOP-11505.003.patch,
> HADOOP-11505.004.patch, HADOOP-11505.005.patch, HADOOP-11505.006.patch,
> HADOOP-11505.007.patch
>
>
> hadoop-mapreduce-client-nativetask fails to use x86 optimizations in some
> cases. Also, on some alternate, non-x86, non-ARM architectures the generated
> code is incorrect. Thanks to Steve Loughran and Edward Nevill for finding
> this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
