Ah so when I built from master it wasn't 1.5. (My mistake,) Nice catch.
On Sat, Feb 13, 2021 at 2:12 AM Kengo Seki <sek...@apache.org> wrote: > Hi Jason, > > The command you executed (./gradlew hadoop-pkg-ind, without any > option) downloads the "trunk" Docker image of CentOS 7 [1] for > building. > That image is based on the master branch. > > On master, we've upgraded Hadoop to 3.x, which requires CMake 3.x to be > built. > CMake 3 is provided as the "cmake3" package on CentOS 7, so we changed > the package dependency for Hadoop in BIGTOP-3472 [2]. > Therefore, the trunk image of CentOS 7 has the "cmake3" package > installed, instead of "cmake". > > On the other hand, on the 1.5.0 branch you're working, building Hadoop > rpm still requires the "cmake" package regardless of distro [3]. > This inconsistency caused your build failure. > > So would you try `./gradlew hadoop-pkg-ind` with a `-Pprefix=1.5.0` option? > It will download the "1.5.0" CentOS 7 image and is supposed to work > with your working branch. :) > > [1]: > https://hub.docker.com/layers/bigtop/slaves/trunk-centos-7/images/sha256-8f5f09663908ac2fce7df11203f1613026cdcd7f52bda7efe608b3f8e9fba8b8 > [2]: > https://github.com/apache/bigtop/commit/e952272ed7aa28d7e2c36b26d81093ca01aa46a8 > [3]: > https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/rpm/hadoop/SPECS/hadoop.spec#L188 > > Kengo Seki <sek...@apache.org> > > On Sat, Feb 13, 2021 at 9:54 AM Jason Wen > <zhenshan....@workday.com.invalid> wrote: > > > > Hi Matt, > > > > Thanks for help. > > This is the result of `ls` of the extracted dir: > > ADVISEADDBEAR:bigtop-1.5.0 zhenshan.wen$ ls > > BUILDING.txt NOTICE bigtop-data-generators > bigtop-tests build.gradle gradlew > provisioner test > > CHANGES.txt README.md bigtop-deploy > bigtop.bom docker output > release.gradle > > LICENSE bigtop-bigpetstore bigtop-packages > bigtop_toolchain docs packages.gradle > settings.gradle > > MAINTAINERS.txt bigtop-ci bigtop-test-framework > build gradle pom.xml src > > > > And this is the command that I used > > ./gradlew hadoop-pkg-ind > > > > I am running this on my mac. > > > > Thanks, > > Jason > > From: Matt Andruff <m...@andruffsolutions.com> > > Reply-To: "dev@bigtop.apache.org" <dev@bigtop.apache.org> > > Date: Friday, February 12, 2021 at 2:07 PM > > To: "dev@bigtop.apache.org" <dev@bigtop.apache.org> > > Subject: Re: Bigtop 1.5 build hadoop package error > > > > so I did finish the build of 1.5 without issues.... but I'm ready to > help figure out why your build didn't work. > > > > Let me know about those asks I made. > > > > On Fri, Feb 12, 2021 at 4:26 PM Matt Andruff <m...@andruffsolutions.com > <mailto:m...@andruffsolutions.com>> wrote: > > Hmm, > > > > I setup ec2 instance, and (./gradlew hadoop-pkg-ind) is humming away... > been building for 7mins.... I'll see if it finishes but appears the docker > image is working as expected. > > > > By chance where you running the command as 'root' ? > > > > On Fri, Feb 12, 2021 at 4:04 PM Matt Andruff <m...@andruffsolutions.com > <mailto:m...@andruffsolutions.com>> wrote: > > Hey Jason, > > > > I'm just spinning up an amazon box to check it out. > > > > Can you supply: > > the exact command you used (include the user) > > `ls` of the extracted dir. > > > > Thanks > > > > Matt > > > > On Fri, Feb 12, 2021 at 3:31 PM Jason Wen <zhenshan....@workday.com.invalid> > wrote: > > I think when I issue './gradlew hadoop-pkg-ind' I am using the pre-built > docker image to build Hadoop packages. > > If that is the case, shouldn't the docker image have the necessary tools > installed like cmake? > > > > Thanks, > > Jason > > > > On 2/12/21, 12:27 PM, "Konstantin Boudnik" <c...@apache.org<mailto: > c...@apache.org>> wrote: > > > > I think here's the reason > > > > error: Failed build dependencies:Installing > > /bigtop/output/hadoop/hadoop-2.10.1-1.el7.src.rpm > > > > cmake is needed by hadoop-2.10.1-1.el7.x86_64 > > > > You need to have a set of tools to be able to build the stack. We are > > doing thing through setting up the toolchain or using a specially > > crafted container, that includes all the toolchain components. > > > > Check this out > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_BIGTOP_How-2Bto-2Bbuild-2BBigtop-2Dtrunk&d=DwIDaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=Y6BHLw6qDSERpiZ37SZEx9Xeqi8kGIrO9FpCUidwlCk&s=JRGSKpSGqaSNjFM4vw2Q3_J7_QeEMq3akC8eJ_VTZnM&e= > > > > -- > > With regards, > > Cos > > > > On 12.02.2021 23:20, Jason Wen wrote: > > > Hi, > > > > > > I am new to bigtop community. I downloaded bigtop 1.5 release and > run ‘./gradlew hadoop-pkg-ind` in the extract dir. But the build fails with > the following error message. The documentation that I followed is > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_BIGTOP_Quickstart-2BGuide-253A-2BBigtop-2BIntegration-2BTest-2BFramework-2B2.0&d=DwIDaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=Y6BHLw6qDSERpiZ37SZEx9Xeqi8kGIrO9FpCUidwlCk&s=TNLFDA7MZnM2ty9fVqJHhrSbl3Ix9DFuwsauzCMVYME&e= > > > > > > Can anyone shed light how to solve the build problem? > > > > > >> Task :hadoop-srpm > > > Caching disabled for task ':hadoop-srpm' because: > > > Build cache is disabled > > > Task ':hadoop-srpm' is not up-to-date because: > > > Task has not declared any outputs despite executing actions. > > > Nothing to do. Exiting... > > > :hadoop-srpm (Thread[Execution worker for ':',5,main]) completed. > Took 0.001 secs. > > > :hadoop-rpm (Thread[Execution worker for ':',5,main]) started. > > > > > >> Task :hadoop-rpm FAILED > > > Caching disabled for task ':hadoop-rpm' because: > > > Build cache is disabled > > > Task ':hadoop-rpm' is not up-to-date because: > > > Task has not declared any outputs despite executing actions. > > > Starting process 'command 'rpmbuild''. Working directory: /bigtop > Command: rpmbuild --define _topdir /bigtop/build/hadoop/rpm/ --define > hadoop_base_version 2.10.1 --define hadoop_version 2.10.1 --define > hadoop_version 2.10.1 --define hadoop_release 1%{?dist} --rebuild > /bigtop/output/hadoop/hadoop-2.10.1-1.el7.src.rpm --define do_maven_deploy > false --define maven_deploy_source false > > > Successfully started process 'command 'rpmbuild'' > > > error: Failed build dependencies:Installing > /bigtop/output/hadoop/hadoop-2.10.1-1.el7.src.rpm > > > > > > cmake is needed by hadoop-2.10.1-1.el7.x86_64 > > > > > > :hadoop-rpm (Thread[Execution worker for ':',5,main]) completed. > Took 0.628 secs. > > > FAILURE: Build failed with an exception. > > > > > > * Where: > > > Script '/bigtop/packages.gradle' line: 507 > > > > > > * What went wrong: > > > Execution failed for task ':hadoop-rpm'. > > >> Process 'command 'rpmbuild'' finished with non-zero exit value 1 > > > > > > * Try: > > > Run with --stacktrace option to get the stack trace. Run with > --debug option to get more log output. Run with --scan to get full insights. > > > > > > * Get more help at > https://urldefense.proofpoint.com/v2/url?u=https-3A__help.gradle.org&d=DwIDaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=Y6BHLw6qDSERpiZ37SZEx9Xeqi8kGIrO9FpCUidwlCk&s=8r0X9nES1diOG-8Y7uYlAa8-w1oXd4yvV0EpCwxN-e8&e= > > > > > > BUILD FAILED in 39s > > > 5 actionable tasks: 5 executed > > > + RESULT=1 > > > + mkdir -p output > > > + docker cp > 89a442c078b2c620285f46b9661c6e1d96df73365b7b2ad8200644dd995c317b:/bigtop/build > . > > > + docker cp > 89a442c078b2c620285f46b9661c6e1d96df73365b7b2ad8200644dd995c317b:/bigtop/output > . > > > + docker rm -f > 89a442c078b2c620285f46b9661c6e1d96df73365b7b2ad8200644dd995c317b > > > 89a442c078b2c620285f46b9661c6e1d96df73365b7b2ad8200644dd995c317b > > > + '[' 1 -ne 0 ']' > > > + exit 1 > > > + docker rm -f > 89a442c078b2c620285f46b9661c6e1d96df73365b7b2ad8200644dd995c317b > > > Error: No such container: > 89a442c078b2c620285f46b9661c6e1d96df73365b7b2ad8200644dd995c317b > > > > > >> Task :hadoop-pkg-ind FAILED > > > > > > FAILURE: Build failed with an exception. > > > > > > * Where: > > > Script > '/Users/zhenshan.wen/code/learning/bigtop-1.5.0/packages.gradle' line: 675 > > > > > > * What went wrong: > > > Execution failed for task ':hadoop-pkg-ind'. > > >> Process 'command 'bash'' finished with non-zero exit value 1 > > > > > > * Try: > > > Run with --stacktrace option to get the stack trace. Run with > --info or --debug option to get more log output. Run with --scan to get > full insights. > > > > > > * Get more help at > https://urldefense.proofpoint.com/v2/url?u=https-3A__help.gradle.org&d=DwIDaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=Y6BHLw6qDSERpiZ37SZEx9Xeqi8kGIrO9FpCUidwlCk&s=8r0X9nES1diOG-8Y7uYlAa8-w1oXd4yvV0EpCwxN-e8&e= > > > > > > BUILD FAILED in 1m 7s > > > 1 actionable task: 1 executed > > > > > > Thanks, > > > Jason > > > >
