Kengo Seki created BIGTOP-3336:
----------------------------------
Summary: Fix Livy build on CentOS and Fedora
Key: BIGTOP-3336
URL: https://issues.apache.org/jira/browse/BIGTOP-3336
Project: Bigtop
Issue Type: Bug
Components: build
Reporter: Kengo Seki
Assignee: Kengo Seki
On CentOS 7, building Livy fails with the following error, due to the lack of
the wheel package.
{code:title=centos-7}
$ ./gradlew livy-pkg
(snip)
[INFO] --- exec-maven-plugin:1.2.1:exec (python-api install) @ livy-python-api
---
/usr/lib/python2.7/site-packages/pkg_resources/py2_warn.py:22: UserWarning:
Setuptools will stop working on Python 2
************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release (no sooner than 2020-04-20).
Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
WARNING: The wheel package is not available.
ERROR: 'pip wheel' requires the 'wheel' package. To fix this, run: pip install
wheel
Traceback (most recent call last):
File "setup.py", line 57, in <module>
tests_require=['pytest']
File "/usr/lib/python2.7/site-packages/setuptools/__init__.py", line 143, in
setup
_install_setup_requires(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/__init__.py", line 138, in
_install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 687, in
fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 783,
in resolve
replace_conflicting=replace_conflicting
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1066,
in best_match
return self.obtain(req, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1078,
in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 743, in
fetch_build_egg
return fetch_build_egg(self, req)
File "/usr/lib/python2.7/site-packages/setuptools/installer.py", line 130, in
fetch_build_egg
raise DistutilsError(str(e))
distutils.errors.DistutilsError: Command '['/usr/bin/python', '-m', 'pip',
'--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpwZ8oio',
'--quiet', 'pytest-runner']' returned non-zero exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Livy Project Parent POM 0.6.0-incubating ........... SUCCESS [ 39.874 s]
[INFO] livy-api ........................................... SUCCESS [ 34.074 s]
[INFO] livy-client-common ................................. SUCCESS [ 4.428 s]
[INFO] livy-test-lib ...................................... SUCCESS [ 11.555 s]
[INFO] livy-rsc ........................................... SUCCESS [ 17.159 s]
[INFO] multi-scala-project-root ........................... SUCCESS [ 0.437 s]
[INFO] livy-core-parent ................................... SUCCESS [ 1.279 s]
[INFO] livy-core_2.11 ..................................... SUCCESS [ 19.109 s]
[INFO] livy-repl-parent ................................... SUCCESS [ 7.326 s]
[INFO] livy-repl_2.11 ..................................... SUCCESS [ 52.145 s]
[INFO] livy-server ........................................ SUCCESS [01:28 min]
[INFO] livy-assembly ...................................... SUCCESS [ 8.376 s]
[INFO] livy-client-http ................................... SUCCESS [ 22.525 s]
[INFO] livy-scala-api-parent .............................. SUCCESS [ 27.709 s]
[INFO] livy-scala-api_2.11 ................................ SUCCESS [ 25.593 s]
[INFO] livy-integration-test .............................. SUCCESS [01:15 min]
[INFO] livy-coverage-report ............................... SUCCESS [ 5.768 s]
[INFO] livy-examples ...................................... SUCCESS [ 25.137 s]
[INFO] livy-python-api 0.6.0-incubating ................... FAILURE [ 1.678 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:48 min
[INFO] Finished at: 2020-04-03T14:56:17Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec
(python-api install) on project livy-python-api: Command execution failed.:
Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :livy-python-api
error: Bad exit status from /var/tmp/rpm-tmp.YCK4qE (%build)
Bad exit status from /var/tmp/rpm-tmp.YCK4qE (%build)
RPM build errors:
> Task :livy-rpm FAILED
FAILURE: Build failed with an exception.
* Where:
Script '/home/vagrant/bigtop/packages.gradle' line: 500
* What went wrong:
Execution failed for task ':livy-rpm'.
> Process 'command 'rpmbuild'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 8m 17s
21 actionable tasks: 21 executed
{code}
In addition, CentOS 8 and Fedora 31 don't have the {{python}} command (they
have the {{python2}} command instead), so the build fails as follows:
{code:title=centos-8}
[INFO] ------------------< org.apache.livy:livy-python-api >-------------------
[INFO] Building livy-python-api 0.6.0-incubating [19/19]
[INFO] --------------------------------[ pom ]---------------------------------
Downloading from central:
https://repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.2.1/exec-maven-plugin-1.2.1.pom
Progress (1): 7.7 kB UTING [9m 19s]
Downloaded from central:
https://repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.2.1/exec-maven-plugin-1.2.1.pom
(7.7 kB at 34 kB/s)
Downloading from central:
https://repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.2.1/exec-maven-plugin-1.2.1.jar
Progress (1): 16/38 kB CUTING [9m 19s]
Progress (1): 33/38 kB
Progress (1): 38 kB
Downloaded from central:
https://repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.2.1/exec-maven-plugin-1.2.1.jar
(38 kB at 163 kB/s)
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (default) @ livy-python-api
---
[INFO] Skipping JaCoCo execution because property jacoco.skip is set.
[INFO] argLine set to empty
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles)
@ livy-python-api ---
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) @
livy-python-api ---
[INFO] No sources to compile
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (python-api install) @ livy-python-api
---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Livy Project Parent POM 0.6.0-incubating ........... SUCCESS [ 51.622 s]
[INFO] livy-api ........................................... SUCCESS [ 32.867 s]
[INFO] livy-client-common ................................. SUCCESS [ 5.211 s]
[INFO] livy-test-lib ...................................... SUCCESS [ 12.630 s]
[INFO] livy-rsc ........................................... SUCCESS [ 16.866 s]
[INFO] multi-scala-project-root ........................... SUCCESS [ 0.628 s]
[INFO] livy-core-parent ................................... SUCCESS [ 1.344 s]
[INFO] livy-core_2.11 ..................................... SUCCESS [ 22.225 s]
[INFO] livy-repl-parent ................................... SUCCESS [ 7.747 s]
[INFO] livy-repl_2.11 ..................................... SUCCESS [01:01 min]
[INFO] livy-server ........................................ SUCCESS [01:38 min]
[INFO] livy-assembly ...................................... SUCCESS [ 8.858 s]
[INFO] livy-client-http ................................... SUCCESS [ 25.649 s]
[INFO] livy-scala-api-parent .............................. SUCCESS [ 29.462 s]
[INFO] livy-scala-api_2.11 ................................ SUCCESS [ 29.101 s]
[INFO] livy-integration-test .............................. SUCCESS [01:37 min]
[INFO] livy-coverage-report ............................... SUCCESS [ 5.936 s]
[INFO] livy-examples ...................................... SUCCESS [ 26.238 s]
[INFO] livy-python-api 0.6.0-incubating ................... FAILURE [ 0.673 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 08:55 min
[INFO] Finished at: 2020-04-03T15:19:40Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec
(python-api install) on project livy-python-api: Command execution failed.:
Cannot run program "python" (in directory
"/home/vagrant/bigtop/build/livy/rpm/BUILD/livy-0.6.0/python-api"): error=2, No
such file or directory -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :livy-python-api
error: Bad exit status from /var/tmp/rpm-tmp.CAmGp1 (%build)
Bad exit status from /var/tmp/rpm-tmp.CAmGp1 (%build)
RPM build errors:
> Task :livy-rpm FAILED
FAILURE: Build failed with an exception.
* Where:
Script '/home/vagrant/bigtop/packages.gradle' line: 500
* What went wrong:
Execution failed for task ':livy-rpm'.
> Process 'command 'rpmbuild'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 9m 24s
21 actionable tasks: 21 executed
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
