Elek, Marton created HDDS-1218:
----------------------------------
Summary: Do the dist-layout-stitching for Ozone after the
test-compile phase
Key: HDDS-1218
URL: https://issues.apache.org/jira/browse/HDDS-1218
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Reporter: Elek, Marton
HDDS-1135 fixed the order of the maven goal executions to include all the
required jar files in the distribution package.
It turned out that the suggested compile phase is too early for the
dist-layout-stitching.
Int case of test-compile execution the shaded ozone datanode service plugin is
not created (it's created at the package phase) but the hadoop-ozone/dist
tries to use (copy it from the source):
The error (from Yetus) is:
{code}
[INFO] --- exec-maven-plugin:1.3.1:exec (dist) @ hadoop-ozone-dist ---
cp: cannot stat
'/testptch/hadoop/hadoop-ozone/objectstore-service/target/hadoop-ozone-objectstore-service-0.4.0-SNAPSHOT-plugin.jar':
No such file or directory
Current directory /testptch/hadoop/hadoop-ozone/dist/target
$ rm -rf ozone-0.4.0-SNAPSHOT
$ mkdir ozone-0.4.0-SNAPSHOT
$ cd ozone-0.4.0-SNAPSHOT
$ cp -p /testptch/hadoop/LICENSE.txt .
$ cp -p /testptch/hadoop/NOTICE.txt .
$ cp -p /testptch/hadoop/README.txt .
$ mkdir -p ./share/hadoop/mapreduce
$ mkdir -p ./share/hadoop/ozone
$ mkdir -p ./share/hadoop/hdds
$ mkdir -p ./share/hadoop/yarn
$ mkdir -p ./share/hadoop/hdfs
$ mkdir -p ./share/hadoop/common
$ mkdir -p ./share/ozone/web
$ mkdir -p ./bin
$ mkdir -p ./sbin
$ mkdir -p ./etc
$ mkdir -p ./libexec
$ cp -r /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/conf
etc/hadoop
$ cp
/testptch/hadoop/hadoop-ozone/dist/src/main/conf/om-audit-log4j2.properties
etc/hadoop
$ cp
/testptch/hadoop/hadoop-ozone/dist/src/main/conf/dn-audit-log4j2.properties
etc/hadoop
$ cp
/testptch/hadoop/hadoop-ozone/dist/src/main/conf/scm-audit-log4j2.properties
etc/hadoop
$ cp /testptch/hadoop/hadoop-ozone/dist/src/main/conf/ozone-site.xml etc/hadoop
$ cp -f /testptch/hadoop/hadoop-ozone/dist/src/main/conf/log4j.properties
etc/hadoop
$ cp /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop
bin/
$ cp
/testptch/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop.cmd
bin/
$ cp /testptch/hadoop/hadoop-ozone/common/src/main/bin/ozone bin/
$ cp
/testptch/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
libexec/
$ cp
/testptch/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd
libexec/
$ cp
/testptch/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
libexec/
$ cp /testptch/hadoop/hadoop-ozone/common/src/main/bin/ozone-config.sh libexec/
$ cp -r /testptch/hadoop/hadoop-ozone/common/src/main/shellprofile.d libexec/
$ cp
/testptch/hadoop/hadoop-common-project/hadoop-common/src/main/bin/hadoop-daemons.sh
sbin/
$ cp
/testptch/hadoop/hadoop-common-project/hadoop-common/src/main/bin/workers.sh
sbin/
$ cp /testptch/hadoop/hadoop-ozone/common/src/main/bin/start-ozone.sh sbin/
$ cp /testptch/hadoop/hadoop-ozone/common/src/main/bin/stop-ozone.sh sbin/
$ mkdir -p ./share/hadoop/ozoneplugin
$ cp
/testptch/hadoop/hadoop-ozone/objectstore-service/target/hadoop-ozone-objectstore-service-0.4.0-SNAPSHOT-plugin.jar
./share/hadoop/ozoneplugin/hadoop-ozone-datanode-plugin-0.4.0-SNAPSHOT.jar
Failed!
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Ozone ................................ SUCCESS [ 5.471 s]
[INFO] Apache Hadoop Ozone Common ......................... SUCCESS [ 13.453 s]
[INFO] Apache Hadoop Ozone Client ......................... SUCCESS [ 6.271 s]
[INFO] Apache Hadoop Ozone Manager Server ................. SUCCESS [ 8.081 s]
[INFO] Apache Hadoop Ozone Object Store REST Service ...... SUCCESS [ 5.535 s]
[INFO] Apache Hadoop Ozone S3 Gateway ..................... SUCCESS [ 7.966 s]
[INFO] Apache Hadoop Ozone Integration Tests .............. SUCCESS [ 7.708 s]
[INFO] Apache Hadoop Ozone FileSystem ..................... SUCCESS [ 7.121 s]
[INFO] Apache Hadoop Ozone FileSystem Single Jar Library .. SUCCESS [ 3.041 s]
[INFO] Apache Hadoop Ozone FileSystem Legacy Jar Library .. SUCCESS [ 3.217 s]
[INFO] Apache Hadoop Ozone Tools .......................... SUCCESS [ 7.720 s]
[INFO] Apache Hadoop Ozone Datanode ....................... SUCCESS [ 1.912 s]
[INFO] Apache Hadoop Ozone Distribution ................... FAILURE [ 2.260 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:21 min
[INFO] Finished at: 2019-03-05T10:22:14+00:00
[INFO] Final Memory: 71M/1852M
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "hdds" could not be activated because it does
not exist.
[WARNING] The requested profile "native" could not be activated because it does
not exist.
[WARNING] The requested profile "yarn-ui" could not be activated because it
does not exist.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec
(dist) on project hadoop-ozone-dist: Command execution failed. Process exited
with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-ozone-dist
{code}
One fix could be to move the dist-layout stitching to a later phase (but before
the tar).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
