Thanks Evans. So let me try solve this first. I submitted this:
https://issues.apache.org/jira/browse/BIGTOP-3204

And I am now working on a test of my fix.

On Thu, Apr 4, 2019 at 2:55 AM Evans Ye <[email protected]> wrote:

> Pay attention on something like no such file of directory. Bigtop’s build
> script relies on the folder layout defined in each component. Typically
> this means the layout has been changed in upstream and we need to do so on
> our end for rpm/deb as well. Usually we’ll take the upstream tarball and
> untar it(or you can find it in bigtop/build directory as well) to see
> what’s the actual layout right now.
>
> Might need to have knowledge on modules in flink so you get the idea what
> needs to be packaged. Let me know if you have a hard time dealing with it.
> I have friend working on Flink who can help.
>
> Evans
>
> Guodong Xu <[email protected]>於 2019年4月3日 週三,下午10:14寫道:
>
> > Hi,
> >
> > I'm trying to upgrade Flink version from 1.4.2 to 1.6.0. What I changed
> is
> > very simple: in bigtop.bom, change
> >      'flink' {
> > ...
> > -      version { base = '1.4.2'; pkg = base; release = 1 }
> > +      version { base = '1.6.0'; pkg = base; release = 1 }
> >
> > Then, I use "./gradlew flink-rpm" to build rpm.
> >
> > However, at the end, I
> >
> > + cp -ra
> >
> >
> '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*'
> >
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/resources/
> > cp: cannot stat
> >
> >
> '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*':
> > No such file or directory
> > error: Bad exit status from /var/tmp/rpm-tmp.mobEKh (%install)
> >
> > Does anybody meet this before? Since I'm new to rpm packaging, I feel
> > difficult in fixing this error. But, if anybody can give me any guidance,
> > I'd love to learn and work for a fix. And any suggestions about which
> file
> > or which folder to look to find a solution, that will be highly
> > appreciated.
> >
> > Thanks a lot. And a more complete log below.
> >
> >
> > -Guodong
> >
> >
> > === a more complete log ===
> >
> >
> > [INFO] Reading assembly descriptor: src/main/assemblies/hadoop.xml
> > [INFO] Copying files to
> > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin
> > [WARNING] Assembly file:
> > /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin
> is
> > not a regular file (it may be a directory). It cannot be attached to the
> > project build for installation or deployment.
> > [INFO]
> > [INFO] --- exec-maven-plugin:1.5.0:exec (create-build-target-link) @
> > flink-dist_2.11 ---
> > [INFO]
> > [INFO] --- maven-surefire-plugin:2.18.1:test (integration-tests) @
> > flink-dist_2.11 ---
> > [INFO] Tests are skipped.
> > [INFO]
> > [INFO] --- maven-install-plugin:2.5.2:install (default-install) @
> > flink-dist_2.11 ---
> > [INFO] Installing
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-dist_2.11-1.6.0.jar
> > to
> >
> >
> /var/lib/jenkins/.m2/repository/org/apache/flink/flink-dist_2.11/1.6.0/flink-dist_2.11-1.6.0.jar
> > [INFO] Installing
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/pom.xml
> > to
> >
> >
> /var/lib/jenkins/.m2/repository/org/apache/flink/flink-dist_2.11/1.6.0/flink-dist_2.11-1.6.0.pom
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] BUILD SUCCESS
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Total time: 32.726 s
> > [INFO] Finished at: 2019-04-03T02:27:17Z
> > [INFO]
> > ------------------------------------------------------------------------
> > + /usr/bin/install -d -m 0755
> > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//etc/rc.d/init.d/
> > + exit 0
> > Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.mobEKh
> > + umask 022
> > + cd /ws/build/flink/rpm//BUILD
> > + '[' /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 '!=' / ']'
> > + rm -rf /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > ++ dirname /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > + mkdir -p /ws/build/flink/rpm/BUILDROOT
> > + mkdir /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > + cd flink-1.6.0
> > + /usr/bin/rm -rf /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > ++ pwd
> > + sh -x /ws/build/flink/rpm/SOURCES/install_flink.sh
> > --prefix=/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > --source-dir=/ws/build/flink/rpm//SOURCES
> >
> >
> --build-dir=/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/
> > + set -e
> > ++ getopt -n /ws/build/flink/rpm/SOURCES/install_flink.sh -o '' -l
> prefix:
> > -l lib-dir: -l installed-lib-dir: -l bin-dir: -l source-dir: -l
> build-dir:
> > -- --prefix=/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > --source-dir=/ws/build/flink/rpm//SOURCES
> >
> >
> --build-dir=/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/
> > + OPTS=' --prefix
> > '\''/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64'\''
> > --source-dir '\''/ws/build/flink/rpm//SOURCES'\'' --build-dir
> >
> >
> '\''/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/'\''
> > --'
> > + '[' 0 '!=' 0 ']'
> > + eval set -- ' --prefix
> > '\''/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64'\''
> > --source-dir '\''/ws/build/flink/rpm//SOURCES'\'' --build-dir
> >
> >
> '\''/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/'\''
> > --'
> > ++ set -- --prefix
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > --source-dir /ws/build/flink/rpm//SOURCES --build-dir
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/
> > --
> > + true
> > + case "$1" in
> > + PREFIX=/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > + shift 2
> > + true
> > + case "$1" in
> > + SOURCE_DIR=/ws/build/flink/rpm//SOURCES
> > + shift 2
> > + true
> > + case "$1" in
> > +
> >
> >
> BUILD_DIR=/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/
> > + shift 2
> > + true
> > + case "$1" in
> > + shift
> > + break
> > + for var in PREFIX BUILD_DIR SOURCE_DIR
> > ++ eval 'echo $PREFIX'
> > +++ echo /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64
> > + '[' -z /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64 ']'
> > + for var in PREFIX BUILD_DIR SOURCE_DIR
> > ++ eval 'echo $BUILD_DIR'
> > +++ echo
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/
> > + '[' -z
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0/
> > ']'
> > + for var in PREFIX BUILD_DIR SOURCE_DIR
> > ++ eval 'echo $SOURCE_DIR'
> > +++ echo /ws/build/flink/rpm//SOURCES
> > + '[' -z /ws/build/flink/rpm//SOURCES ']'
> > + '[' -f /ws/build/flink/rpm//SOURCES/bigtop.bom ']'
> > + . /ws/build/flink/rpm//SOURCES/bigtop.bom
> > ++ ZOOKEEPER_VERSION=3.4.6
> > ++ HADOOP_VERSION=2.8.4
> > ++ IGNITE_HADOOP_VERSION=1.9.0
> > ++ HBASE_VERSION=1.3.2
> > ++ HIVE_VERSION=2.3.3
> > ++ TEZ_VERSION=0.9.1
> > ++ SQOOP_VERSION=1.4.6
> > ++ SQOOP2_VERSION=1.99.4
> > ++ MAHOUT_VERSION=0.13.0
> > ++ FLUME_VERSION=1.8.0
> > ++ GIRAPH_VERSION=1.1.0
> > ++ DATAFU_VERSION=1.3.0
> > ++ SOLR_VERSION=6.6.0
> > ++ CRUNCH_VERSION=0.15.0
> > ++ SPARK1_VERSION=1.6.2
> > ++ SPARK_VERSION=2.2.1
> > ++ FLINK_VERSION=1.6.0
> > ++ PHOENIX_VERSION=4.13.1-HBase-1.3
> > ++ BIGTOP_GROOVY_VERSION=2.4.10
> > ++ BIGTOP_UTILS_VERSION=1.3.0
> > ++ BIGTOP_JSVC_VERSION=1.0.15
> > ++ BIGTOP_TOMCAT_VERSION=6.0.45
> > ++ ALLUXIO_VERSION=1.0.1
> > ++ KAFKA_VERSION=0.10.2.2
> > ++ YCSB_VERSION=0.12.0
> > ++ HAMA_VERSION=0.7.1
> > ++ ZEPPELIN_VERSION=0.7.3
> > ++ TAJO_VERSION=0.11.1
> > ++ APEX_VERSION=3.6.0
> > ++ QFS_VERSION=2.0.0
> > ++ GPDB_VERSION=5.10.0
> > ++ AMBARI_VERSION=2.6.1
> > ++ JDK_VERSION=1.8
> > ++ SCALA_VERSION=2.11.8
> > + LIB_DIR=/usr/lib/flink
> > + INSTALLED_LIB_DIR=/usr/lib/flink
> > + BIN_DIR=/usr/bin
> > + CONF_DIR=/etc/flink/conf.dist
> > + install -d -m 0755
> > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink
> > + install -d -m 0755
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/bin
> > + install -d -m 0755
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/lib
> > + install -d -m 0755
> >
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/examples
> > + install -d -m 0755
> >
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/resources
> > + install -d -m 0755
> >
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//etc/flink/conf.dist
> > + install -d -m 0755
> > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64/var/log/flink
> > + install -d -m 0755
> > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64/var/log/flink-cli
> > + install -d -m 0755
> > /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64/var/run/flink
> > + cp -ra
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/flink-dist_2.11-1.6.0.jar
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/flink-python_2.11-1.6.0.jar
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/flink-shaded-hadoop2-uber-1.6.0.jar
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/log4j-1.2.17.jar
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//lib/slf4j-log4j12-1.7.7.jar
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/lib/
> > + cp -a
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/config.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink-console.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink-daemon.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/flink.bat
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/historyserver.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/jobmanager.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/mesos-appmaster-job.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/mesos-appmaster.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/mesos-taskmanager.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/pyflink-stream.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/pyflink.bat
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/pyflink.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/sql-client.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/standalone-job.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-cluster.bat
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-cluster.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-scala-shell.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/start-zookeeper-quorum.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/stop-cluster.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/stop-zookeeper-quorum.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/taskmanager.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/yarn-session.sh
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//bin/zookeeper.sh
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/bin/
> > + rm -rf
> >
> >
> '/ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/bin/*.cmd'
> > + rm -rf
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/log
> > + cp -a
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/flink-conf.yaml
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j-cli.properties
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j-console.properties
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j-yarn-session.properties
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/log4j.properties
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/logback-console.xml
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/logback-yarn.xml
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/logback.xml
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/masters
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/slaves
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/sql-client-defaults.yaml
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//conf/zoo.cfg
> >
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//etc/flink/conf.dist
> > + ln -s /etc/flink/conf
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/conf
> > + cp -ra
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/batch
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/gelly
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/python
> >
> >
> /ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//examples/streaming
> >
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/examples/
> > + cp -ra
> >
> >
> '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*'
> >
> >
> /ws/build/flink/rpm/BUILDROOT/flink-1.6.0-1.el7.aarch64//usr/lib/flink/resources/
> > cp: cannot stat
> >
> >
> '/ws/build/flink/rpm/BUILD/flink-1.6.0/flink-dist/target/flink-1.6.0-bin/flink-1.6.0//resources/*':
> > No such file or directory
> > error: Bad exit status from /var/tmp/rpm-tmp.mobEKh (%install)
> >
> >
> > RPM build errors:
> >     Bad exit status from /var/tmp/rpm-tmp.mobEKh (%install)
> > :flink-rpm FAILED
> >
> > FAILURE: Build failed with an exception.
> >
> > * Where:
> > Script '/ws/packages.gradle' line: 457
> >
> > * What went wrong:
> > Execution failed for task ':flink-rpm'.
> > > Process 'command 'rpmbuild'' finished with non-zero exit value 1
> >
> > * Try:
> > Run with --stacktrace option to get the stack trace. Run with --info or
> > --debug option to get more log output.
> >
> > * Get more help at https://help.gradle.org
> >
> > BUILD FAILED in 29m 54s
> > 5 actionable tasks: 5 executed
> >
>

Reply via email to