I was able to build spark-rpm with exactly same command you provided. Maybe you can check the sha256 of docker image. Mine is 059ac23e10e7.
2017-10-27 14:14 GMT+08:00 Konstantin Boudnik <[email protected]>: > Ah great! I was about to start digging into our CI settings ;) > What wass the limit of RAM that fixed the issue, BTW? > -- > With regards, > Konstantin (Cos) Boudnik > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622 > > Disclaimer: Opinions expressed in this email are those of the author, > and do not necessarily represent the views of any company the author > might be affiliated with at the moment of writing. > > > On Thu, Oct 26, 2017 at 10:13 PM, Chi Zewen <[email protected]> wrote: > > Hi Cos, > > > > I’ve solved this problem by modifying the memory limit of docker. I > really don’t know what I would have done without your help. Thanks a lot. I > really appreciate it. > > > > Regards, > > Zewen > > > > > >> 在 2017年10月26日，下午12:28，Konstantin Boudnik <[email protected]> 写道： > >> > >> Are you using our recommended toolchain? Or the readily available > docker images we use for the CI builds? > >> -- > >> Regards, > >> Cos > >> > >> On October 25, 2017 8:06:34 PM PDT, Chi Zewen <[email protected]> > wrote: > >>> Hi, > >>> > >>> Nice to meet you all. I’m Zewen, a trainee working in China. > >>> > >>> I’m trying to package spark into rpm packages by using “./gradlew” & > >>> “gradle spark-rpm” on docker(centos7). > >>> I tried many times and failed with the following output: > >>> > >>> [INFO] Compiling 26 source files to > >>> /root/host/runtime/bigtop-rel-1.2.0/build/spark/rpm/BUILD/ > spark-2.1.0/yarn/target/scala-2.11/classes > >>> at 1508926971435 > >>> [WARNING] warning: Class org.mortbay.jetty.Server not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not > >>> found - continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.Connector not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class > >>> org.mortbay.jetty.handler.ContextHandlerCollection not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - > >>> continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not > >>> found - continuing with a stub. > >>> [WARNING] warning: Class org.mortbay.jetty.Connector not found - > >>> continuing with a stub. > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] Reactor Summary: > >>> [INFO] > >>> [INFO] > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] Skipping Spark Project External Flume > >>> [INFO] This project has been banned from the build due to previous > >>> failures. > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] Skipping Spark Integration for Kafka 0.8 > >>> [INFO] This project has been banned from the build due to previous > >>> failures. > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] Skipping Spark Integration for Kafka 0.10 > >>> [INFO] This project has been banned from the build due to previous > >>> failures. > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] Spark Project Parent POM ........................... SUCCESS [ > >>> 5.967 s] > >>> [INFO] Spark Project Tags ................................. SUCCESS [ > >>> 9.337 s] > >>> [INFO] Spark Project Sketch ............................... SUCCESS [ > >>> 24.555 s] > >>> [INFO] Spark Project Networking ........................... SUCCESS [ > >>> 26.167 s] > >>> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ > >>> 18.151 s] > >>> [INFO] Spark Project Unsafe ............................... SUCCESS [ > >>> 32.972 s] > >>> [INFO] Spark Project Launcher ............................. SUCCESS [ > >>> 20.188 s] > >>> [INFO] Spark Project Core ................................. SUCCESS > >>> [07:24 min] > >>> [INFO] Spark Project ML Local Library ..................... SUCCESS > >>> [01:10 min] > >>> [INFO] Spark Project GraphX ............................... SUCCESS > >>> [01:59 min] > >>> [INFO] Spark Project Streaming ............................ SUCCESS > >>> [04:01 min] > >>> [INFO] Spark Project Catalyst ............................. FAILURE > >>> [03:37 min] > >>> [INFO] Spark Project SQL .................................. SKIPPED > >>> [INFO] Spark Project ML Library ........................... SKIPPED > >>> [INFO] Spark Project Tools ................................ SUCCESS [ > >>> 14.261 s] > >>> [INFO] Spark Project Hive ................................. SKIPPED > >>> [INFO] Spark Project REPL ................................. SKIPPED > >>> [INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ > >>> 15.728 s] > >>> [INFO] Spark Project YARN ................................. SKIPPED > >>> [INFO] Spark Project Hive Thrift Server ................... SKIPPED > >>> [INFO] Spark Project Assembly ............................. SKIPPED > >>> [INFO] Spark Project External Flume Sink .................. SUCCESS [ > >>> 36.498 s] > >>> [INFO] Spark Project External Flume ....................... SKIPPED > >>> [INFO] Spark Project External Flume Assembly .............. SKIPPED > >>> [INFO] Spark Integration for Kafka 0.8 .................... SKIPPED > >>> [INFO] Spark Project Examples ............................. SKIPPED > >>> [INFO] Spark Project External Kafka Assembly .............. SKIPPED > >>> [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED > >>> [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED > >>> [INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] BUILD FAILURE > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [INFO] Total time: 15:09 min (Wall Clock) > >>> [INFO] Finished at: 2017-10-25T10:24:20+00:00 > >>> [INFO] Final Memory: 64M/683M > >>> [INFO] > >>> ------------------------------------------------------------ > ------------ > >>> [ERROR] Failed to execute goal > >>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile > >>> (scala-compile-first) on project spark-catalyst_2.11: wrap: > >>> org.apache.commons.exec.ExecuteException: Process exited with an > error: > >>> 137 (Exit value: 137) -> [Help 1] > >>> [ERROR] > >>> [ERROR] To see the full stack trace of the errors, re-run Maven with > >>> the -e switch. > >>> [ERROR] Re-run Maven using the -X switch to enable full debug logging. > >>> [ERROR] > >>> [ERROR] For more information about the errors and possible solutions, > >>> please read the following articles: > >>> [ERROR] [Help 1] > >>> http://cwiki.apache.org/confluence/display/MAVEN/ > MojoExecutionException > >>> [ERROR] > >>> [ERROR] After correcting the problems, you can resume the build with > >>> the command > >>> [ERROR] mvn <goals> -rf :spark-catalyst_2.11 > >>> > >>> > >>> RPM build errors: > >>> error: Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build) > >>> Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build) > >>> :spark-rpm FAILED > >>> > >>> FAILURE: Build failed with an exception. > >>> > >>> * Where: > >>> Script '/root/host/runtime/bigtop-rel-1.2.0/packages.gradle' line: 462 > >>> > >>> * What went wrong: > >>> Execution failed for task ':spark-rpm'. > >>>> Process 'command 'rpmbuild'' finished with non-zero exit value 1 > >>> > >>> * Try: > >>> Run with --stacktrace option to get the stack trace. Run with --info or > >>> --debug option to get more log output. > >>> > >>> BUILD FAILED > >>> > >>> Thanks, > >>> > >>> Zewen Chi > > >
