[
https://issues.apache.org/jira/browse/FLINK-14306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941799#comment-16941799
]
Piotr Nowojski commented on FLINK-14306:
----------------------------------------
After manually installing {{setuptools}} and {{python-pip}} on the machine I'm
getting a new error:
{noformat}
[INFO] --- exec-maven-plugin:1.5.0:exec (Protos Generation) @ flink-python_2.11
---
/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/gen_protos.py:49:
UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into
/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/../.eggs/grpcio-wheels
ERROR: Can not combine '--user' and '--prefix' as they imply different
installation locations
You are using pip version 8.1.1, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File
"/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/gen_protos.py",
line 126, in _install_grpcio_tools_and_generate_proto_files
'--upgrade', GRPC_TOOLS, "-I"])
File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/usr/bin/python', '-m', 'pip', 'install',
'--prefix',
'/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/../.eggs/grpcio-wheels',
'--build',
'/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/../.eggs/grpcio-wheels-build',
'--upgrade', 'grpcio-tools>=1.3.5,<=1.14.2', '-I']' returned non-zero exit
status 1
Traceback (most recent call last):
File
"/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/gen_protos.py",
line 146, in <module>
generate_proto_files(force=True)
File
"/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/gen_protos.py",
line 91, in generate_proto_files
raise ValueError("Proto generation failed (see log for details).")
ValueError: Proto generation failed (see log for details).
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit
value: 1)
at
org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:764)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:711)
at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:289)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
{noformat}
> flink-python build fails with No module named pkg_resources
> -----------------------------------------------------------
>
> Key: FLINK-14306
> URL: https://issues.apache.org/jira/browse/FLINK-14306
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 1.10.0
> Reporter: Piotr Nowojski
> Priority: Critical
> Fix For: 1.10.0
>
>
> [Benchmark
> builds|http://codespeed.dak8s.net:8080/job/flink-master-benchmarks/4576/console]
> started to fail with
> {noformat}
> [INFO] Adding generated sources (java):
> /home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/target/generated-sources
> [INFO]
> [INFO] --- exec-maven-plugin:1.5.0:exec (Protos Generation) @
> flink-python_2.11 ---
> Traceback (most recent call last):
> File
> "/home/jenkins/workspace/flink-master-benchmarks/flink/flink-python/pyflink/gen_protos.py",
> line 33, in <module>
> import pkg_resources
> ImportError: No module named pkg_resources
> [ERROR] Command execution failed.
> (...)
> [INFO] flink-state-processor-api .......................... SUCCESS [ 0.299
> s]
> [INFO] flink-python ....................................... FAILURE [ 0.434
> s]
> [INFO] flink-scala-shell .................................. SKIPPED
> {noformat}
> because of this ticket: https://issues.apache.org/jira/browse/FLINK-14018
> I think I can solve the benchmark builds failing quite easily by installing
> {{setuptools}} python package, so this ticket is not about this, but about
> deciding how should we treat such kind of external dependencies. I don't see
> this dependency being mentioned anywhere in the documentation ([for example
> here|https://ci.apache.org/projects/flink/flink-docs-stable/flinkDev/building.html]).
> Probably at the very least those external dependencies should be documented,
> but also I fear about such kind of manual steps to do before building the
> Flink can become a problem if would grow out of control. Some questions:
> # Do we really need this dependency?
> # Could this dependency be resolve automatically? By installing into a local
> python virtual environment?
> # Should we document those dependencies somewhere?
> # Maybe we should not build flink-python by default?
> CC [~hequn] [~dian.fu] [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
