[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-13297:
---------------------------------
Status: Patch Available (was: In Progress)
The problem is that hadoop-project defines a use of the remote-resources-plugin
that needs the hadoop-build-tool jar but that plugin declaration does not
declare the dependency. You can check this by running {{mvn validate}} to look
at the reactor order. Here you can see that the Hadoop Build Tools module is
way at the end, because Maven doesn’t think anything needs it:
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache Hadoop Main
[INFO] Apache Hadoop Project POM
[INFO] Apache Hadoop Annotations
[INFO] Apache Hadoop Project Dist POM
[INFO] Apache Hadoop Assemblies
[INFO] Apache Hadoop Maven Plugins
[INFO] Apache Hadoop MiniKDC
[INFO] Apache Hadoop Auth
[INFO] Apache Hadoop Auth Examples
[INFO] Apache Hadoop Common
[INFO] Apache Hadoop NFS
[INFO] Apache Hadoop KMS
[INFO] Apache Hadoop Common Project
[INFO] Apache Hadoop HDFS
[INFO] Apache Hadoop HttpFS
[INFO] Apache Hadoop HDFS BookKeeper Journal
[INFO] Apache Hadoop HDFS-NFS
[INFO] Apache Hadoop HDFS Project
[INFO] hadoop-yarn
[INFO] hadoop-yarn-api
[INFO] hadoop-yarn-common
[INFO] hadoop-yarn-server
[INFO] hadoop-yarn-server-common
[INFO] hadoop-yarn-server-nodemanager
[INFO] hadoop-yarn-server-web-proxy
[INFO] hadoop-yarn-server-applicationhistoryservice
[INFO] hadoop-yarn-server-resourcemanager
[INFO] hadoop-yarn-server-tests
[INFO] hadoop-yarn-client
[INFO] hadoop-yarn-applications
[INFO] hadoop-yarn-applications-distributedshell
[INFO] hadoop-yarn-applications-unmanaged-am-launcher
[INFO] hadoop-yarn-site
[INFO] hadoop-yarn-registry
[INFO] hadoop-yarn-project
[INFO] hadoop-mapreduce-client
[INFO] hadoop-mapreduce-client-core
[INFO] hadoop-mapreduce-client-common
[INFO] hadoop-mapreduce-client-shuffle
[INFO] hadoop-mapreduce-client-app
[INFO] hadoop-mapreduce-client-hs
[INFO] hadoop-mapreduce-client-jobclient
[INFO] hadoop-mapreduce-client-hs-plugins
[INFO] Apache Hadoop MapReduce Examples
[INFO] hadoop-mapreduce
[INFO] Apache Hadoop MapReduce Streaming
[INFO] Apache Hadoop Distributed Copy
[INFO] Apache Hadoop Archives
[INFO] Apache Hadoop Rumen
[INFO] Apache Hadoop Gridmix
[INFO] Apache Hadoop Data Join
[INFO] Apache Hadoop Ant Tasks
[INFO] Apache Hadoop Extras
[INFO] Apache Hadoop Pipes
[INFO] Apache Hadoop OpenStack support
[INFO] Apache Hadoop Amazon Web Services support
[INFO] Apache Hadoop Client
[INFO] Apache Hadoop Mini-Cluster
[INFO] Apache Hadoop Scheduler Load Simulator
[INFO] Apache Hadoop Tools Dist
[INFO] Apache Hadoop Tools
[INFO] Apache Hadoop Distribution
[INFO] Apache Hadoop Build Tools
{code}
after patch on branch-2.6, we can see that the build tools module happens right
away, as it should:
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache Hadoop Main
[INFO] Apache Hadoop Build Tools
[INFO] Apache Hadoop Project POM
[INFO] Apache Hadoop Annotations
[INFO] Apache Hadoop Project Dist POM
[INFO] Apache Hadoop Assemblies
[INFO] Apache Hadoop Maven Plugins
[INFO] Apache Hadoop MiniKDC
[INFO] Apache Hadoop Auth
[INFO] Apache Hadoop Auth Examples
[INFO] Apache Hadoop Common
[INFO] Apache Hadoop NFS
[INFO] Apache Hadoop KMS
[INFO] Apache Hadoop Common Project
[INFO] Apache Hadoop HDFS
[INFO] Apache Hadoop HttpFS
[INFO] Apache Hadoop HDFS BookKeeper Journal
[INFO] Apache Hadoop HDFS-NFS
[INFO] Apache Hadoop HDFS Project
[INFO] hadoop-yarn
[INFO] hadoop-yarn-api
[INFO] hadoop-yarn-common
[INFO] hadoop-yarn-server
[INFO] hadoop-yarn-server-common
[INFO] hadoop-yarn-server-nodemanager
[INFO] hadoop-yarn-server-web-proxy
[INFO] hadoop-yarn-server-applicationhistoryservice
[INFO] hadoop-yarn-server-resourcemanager
[INFO] hadoop-yarn-server-tests
[INFO] hadoop-yarn-client
[INFO] hadoop-yarn-applications
[INFO] hadoop-yarn-applications-distributedshell
[INFO] hadoop-yarn-applications-unmanaged-am-launcher
[INFO] hadoop-yarn-site
[INFO] hadoop-yarn-registry
[INFO] hadoop-yarn-project
[INFO] hadoop-mapreduce-client
[INFO] hadoop-mapreduce-client-core
[INFO] hadoop-mapreduce-client-common
[INFO] hadoop-mapreduce-client-shuffle
[INFO] hadoop-mapreduce-client-app
[INFO] hadoop-mapreduce-client-hs
[INFO] hadoop-mapreduce-client-jobclient
[INFO] hadoop-mapreduce-client-hs-plugins
[INFO] Apache Hadoop MapReduce Examples
[INFO] hadoop-mapreduce
[INFO] Apache Hadoop MapReduce Streaming
[INFO] Apache Hadoop Distributed Copy
[INFO] Apache Hadoop Archives
[INFO] Apache Hadoop Rumen
[INFO] Apache Hadoop Gridmix
[INFO] Apache Hadoop Data Join
[INFO] Apache Hadoop Ant Tasks
[INFO] Apache Hadoop Extras
[INFO] Apache Hadoop Pipes
[INFO] Apache Hadoop OpenStack support
[INFO] Apache Hadoop Amazon Web Services support
[INFO] Apache Hadoop Client
[INFO] Apache Hadoop Mini-Cluster
[INFO] Apache Hadoop Scheduler Load Simulator
[INFO] Apache Hadoop Tools Dist
[INFO] Apache Hadoop Tools
[INFO] Apache Hadoop Distribution
{code}
This same change should go into master and all branches that use the
remote-resources-plugin, though the immediate symptom is masked on 2.8+ due to
HADOOP-11889
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
> ----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 2.7.3, 2.6.5
> Reporter: Akira Ajisaka
> Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
