Thanks Evans. So let me try solve this first. I submitted this: https://issues.apache.org/jira/browse/BIGTOP-3204
And I am now working on a test of my fix. On Thu, Apr 4, 2019 at 2:55 AM Evans Ye <[email protected]> wrote: > Pay attention on something like no such file of directory. Bigtop’s build > script relies on the folder layout defined in each component. Typically > this means the layout has been changed in upstream and we need to do so on > our end for rpm/deb as well. Usually we’ll take the upstream tarball and > untar it(or you can find it in bigtop/build directory as well) to see > what’s the actual layout right now. > > Might need to have knowledge on modules in flink so you get the idea what > needs to be packaged. Let me know if you have a hard time dealing with it. > I have friend working on Flink who can help. > > Evans > > Guodong Xu <[email protected]>於 2019年4月3日 週三,下午10:14寫道: > > > Hi, > > > > I'm trying to upgrade Flink version from 1.4.2 to 1.6.0. What I changed > is > > very simple: in bigtop.bom, change > > 'flink' { > > ... > > - version { base = '1.4.2'; pkg = base; release = 1 } > > + version { base = '1.6.0'; pkg = base; release = 1 } > > > > Then, I use "./gradlew flink-rpm" to build rpm. > > > > However, at the end, I > > > > + cp -ra > > > > > '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*' > > > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/resources/ > > cp: cannot stat > > > > > '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*': > > No such file or directory > > error: Bad exit status from /var/tmp/rpm-tmp.mobEKh (%install) > > > > Does anybody meet this before? Since I'm new to rpm packaging, I feel > > difficult in fixing this error. But, if anybody can give me any guidance, > > I'd love to learn and work for a fix. And any suggestions about which > file > > or which folder to look to find a solution, that will be highly > > appreciated. > > > > Thanks a lot. And a more complete log below. > > > > > > -Guodong > > > > > > === a more complete log === > > > > > > [INFO] Reading assembly descriptor: src/main/assemblies/hadoop.xml > > [INFO] Copying files to > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin > > [WARNING] Assembly file: > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin > is > > not a regular file (it may be a directory). It cannot be attached to the > > project build for installation or deployment. > > [INFO] > > [INFO] --- exec-maven-plugin:1.5.0:exec (create-build-target-link) @ > > flink-dist_2.11 --- > > [INFO] > > [INFO] --- maven-surefire-plugin:2.18.1:test (integration-tests) @ > > flink-dist_2.11 --- > > [INFO] Tests are skipped. > > [INFO] > > [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ > > flink-dist_2.11 --- > > [INFO] Installing > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-dist_2.11-1.6.0.jar > > to > > > > > /var/lib/jenkins/.m2/repository/org/apache/flink/flink-dist_2.11/1.6.0/flink-dist_2.11-1.6.0.jar > > [INFO] Installing > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/pom.xml > > to > > > > > /var/lib/jenkins/.m2/repository/org/apache/flink/flink-dist_2.11/1.6.0/flink-dist_2.11-1.6.0.pom > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] BUILD SUCCESS > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Total time: 32.726 s > > [INFO] Finished at: 2019-04-03T02:27:17Z > > [INFO] > > ------------------------------------------------------------------------ > > + /usr/bin/install -d -m 0755 > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//etc/rc.d/init.d/ > > + exit 0 > > Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.mobEKh > > + umask 022 > > + cd /ws/build/flink/rpm//BUILD > > + '[' /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 '!=' / ']' > > + rm -rf /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > ++ dirname /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > + mkdir -p /ws/build/flink/rpm/BUILDROOT > > + mkdir /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > + cd flink-1.6.0 > > + /usr/bin/rm -rf /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > ++ pwd > > + sh -x /ws/build/flink/rpm/SOURCES/install_flink.sh > > --prefix=/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > --source-dir=/ws/build/flink/rpm//SOURCES > > > > > --build-dir=/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/ > > + set -e > > ++ getopt -n /ws/build/flink/rpm/SOURCES/install_flink.sh -o '' -l > prefix: > > -l lib-dir: -l installed-lib-dir: -l bin-dir: -l source-dir: -l > build-dir: > > -- --prefix=/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > --source-dir=/ws/build/flink/rpm//SOURCES > > > > > --build-dir=/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/ > > + OPTS=' --prefix > > '\''/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64'\'' > > --source-dir '\''/ws/build/flink/rpm//SOURCES'\'' --build-dir > > > > > '\''/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/'\'' > > --' > > + '[' 0 '!=' 0 ']' > > + eval set -- ' --prefix > > '\''/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64'\'' > > --source-dir '\''/ws/build/flink/rpm//SOURCES'\'' --build-dir > > > > > '\''/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/'\'' > > --' > > ++ set -- --prefix > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > --source-dir /ws/build/flink/rpm//SOURCES --build-dir > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/ > > -- > > + true > > + case "$1" in > > + PREFIX=/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > + shift 2 > > + true > > + case "$1" in > > + SOURCE_DIR=/ws/build/flink/rpm//SOURCES > > + shift 2 > > + true > > + case "$1" in > > + > > > > > BUILD_DIR=/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/ > > + shift 2 > > + true > > + case "$1" in > > + shift > > + break > > + for var in PREFIX BUILD_DIR SOURCE_DIR > > ++ eval 'echo $PREFIX' > > +++ echo /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 > > + '[' -z /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 ']' > > + for var in PREFIX BUILD_DIR SOURCE_DIR > > ++ eval 'echo $BUILD_DIR' > > +++ echo > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/ > > + '[' -z > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/ > > ']' > > + for var in PREFIX BUILD_DIR SOURCE_DIR > > ++ eval 'echo $SOURCE_DIR' > > +++ echo /ws/build/flink/rpm//SOURCES > > + '[' -z /ws/build/flink/rpm//SOURCES ']' > > + '[' -f /ws/build/flink/rpm//SOURCES/bigtop.bom ']' > > + . /ws/build/flink/rpm//SOURCES/bigtop.bom > > ++ ZOOKEEPER_VERSION=3.4.6 > > ++ HADOOP_VERSION=2.8.4 > > ++ IGNITE_HADOOP_VERSION=1.9.0 > > ++ HBASE_VERSION=1.3.2 > > ++ HIVE_VERSION=2.3.3 > > ++ TEZ_VERSION=0.9.1 > > ++ SQOOP_VERSION=1.4.6 > > ++ SQOOP2_VERSION=1.99.4 > > ++ MAHOUT_VERSION=0.13.0 > > ++ FLUME_VERSION=1.8.0 > > ++ GIRAPH_VERSION=1.1.0 > > ++ DATAFU_VERSION=1.3.0 > > ++ SOLR_VERSION=6.6.0 > > ++ CRUNCH_VERSION=0.15.0 > > ++ SPARK1_VERSION=1.6.2 > > ++ SPARK_VERSION=2.2.1 > > ++ FLINK_VERSION=1.6.0 > > ++ PHOENIX_VERSION=4.13.1-HBase-1.3 > > ++ BIGTOP_GROOVY_VERSION=2.4.10 > > ++ BIGTOP_UTILS_VERSION=1.3.0 > > ++ BIGTOP_JSVC_VERSION=1.0.15 > > ++ BIGTOP_TOMCAT_VERSION=6.0.45 > > ++ ALLUXIO_VERSION=1.0.1 > > ++ KAFKA_VERSION=0.10.2.2 > > ++ YCSB_VERSION=0.12.0 > > ++ HAMA_VERSION=0.7.1 > > ++ ZEPPELIN_VERSION=0.7.3 > > ++ TAJO_VERSION=0.11.1 > > ++ APEX_VERSION=3.6.0 > > ++ QFS_VERSION=2.0.0 > > ++ GPDB_VERSION=5.10.0 > > ++ AMBARI_VERSION=2.6.1 > > ++ JDK_VERSION=1.8 > > ++ SCALA_VERSION=2.11.8 > > + LIB_DIR=/usr/lib/flink > > + INSTALLED_LIB_DIR=/usr/lib/flink > > + BIN_DIR=/usr/bin > > + CONF_DIR=/etc/flink/conf.dist > > + install -d -m 0755 > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink > > + install -d -m 0755 > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/bin > > + install -d -m 0755 > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/lib > > + install -d -m 0755 > > > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/examples > > + install -d -m 0755 > > > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/resources > > + install -d -m 0755 > > > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//etc/flink/conf.dist > > + install -d -m 0755 > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64/var/log/flink > > + install -d -m 0755 > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64/var/log/flink-cli > > + install -d -m 0755 > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64/var/run/flink > > + cp -ra > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/flink-dist_2.11-1.6.0.jar > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/flink-python_2.11-1.6.0.jar > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/flink-shaded-hadoop2-uber-1.6.0.jar > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/log4j-1.2.17.jar > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/slf4j-log4j12-1.7.7.jar > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/lib/ > > + cp -a > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/config.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink-console.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink-daemon.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink.bat > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/historyserver.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/jobmanager.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/mesos-appmaster-job.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/mesos-appmaster.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/mesos-taskmanager.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/pyflink-stream.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/pyflink.bat > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/pyflink.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/sql-client.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/standalone-job.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-cluster.bat > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-cluster.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-scala-shell.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-zookeeper-quorum.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/stop-cluster.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/stop-zookeeper-quorum.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/taskmanager.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/yarn-session.sh > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/zookeeper.sh > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/bin/ > > + rm -rf > > > > > '/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/bin/*.cmd' > > + rm -rf > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/log > > + cp -a > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/flink-conf.yaml > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j-cli.properties > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j-console.properties > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j-yarn-session.properties > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j.properties > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/logback-console.xml > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/logback-yarn.xml > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/logback.xml > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/masters > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/slaves > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/sql-client-defaults.yaml > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/zoo.cfg > > > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//etc/flink/conf.dist > > + ln -s /etc/flink/conf > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/conf > > + cp -ra > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/batch > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/gelly > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/python > > > > > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/streaming > > > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/examples/ > > + cp -ra > > > > > '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*' > > > > > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/resources/ > > cp: cannot stat > > > > > '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*': > > No such file or directory > > error: Bad exit status from /var/tmp/rpm-tmp.mobEKh (%install) > > > > > > RPM build errors: > > Bad exit status from /var/tmp/rpm-tmp.mobEKh (%install) > > :flink-rpm FAILED > > > > FAILURE: Build failed with an exception. > > > > * Where: > > Script '/ws/packages.gradle' line: 457 > > > > * What went wrong: > > Execution failed for task ':flink-rpm'. > > > Process 'command 'rpmbuild'' finished with non-zero exit value 1 > > > > * Try: > > Run with --stacktrace option to get the stack trace. Run with --info or > > --debug option to get more log output. > > > > * Get more help at https://help.gradle.org > > > > BUILD FAILED in 29m 54s > > 5 actionable tasks: 5 executed > > >
