[
https://issues.apache.org/jira/browse/BEAM-8979?focusedWorklogId=380254&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380254
]
ASF GitHub Bot logged work on BEAM-8979:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Jan/20 23:12
Start Date: 31/Jan/20 23:12
Worklog Time Spent: 10m
Work Description: udim commented on issue #10734: [BEAM-8979] reintroduce
mypy-protobuf stub generation
URL: https://github.com/apache/beam/pull/10734#issuecomment-580951255
> @udim I can't trigger the jenkins jobs. Also, it's unclear to me how to
run this job locally via gradle. Can you give me some advice?
>
> Also, do you have any idea what's different about the way these two jobs
are run that would cause their virtualenv's bin dir to not be on the search
PATH?
I ran:
```
$ git fetch origin pull/10734/head:pr10734
$ git checkout pr10734
$ ./gradlew :sdks:python:sdist
...
> Task :sdks:python:sdist
setup.py:244: UserWarning: You are using Apache Beam with Python 2. New
releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
/usr/local/google/home/ehudm/src/beam/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/dist.py:476:
UserWarning: Normalizing '2.20.0.dev' to '2.20.0.dev0'
normalized_version,
No handlers could be found for logger "gen_protos"
Traceback (most recent call last):
File "setup.py", line 308, in <module>
'mypy': generate_protos_first(mypy),
File
"/usr/local/google/home/ehudm/src/beam/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/__init__.py",
line 145, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File
"/usr/local/google/home/ehudm/src/beam/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/command/sdist.py",
line 44, in run
self.run_command('egg_info')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 232, in run
gen_protos.generate_proto_files()
File "/usr/local/google/home/ehudm/src/beam/sdks/python/gen_protos.py",
line 314, in generate_proto_files
protoc_gen_mypy = _find_protoc_gen_mypy()
File "/usr/local/google/home/ehudm/src/beam/sdks/python/gen_protos.py",
line 233, in _find_protoc_gen_mypy
(fname, ', '.join(search_paths)))
RuntimeError: Could not find protoc-gen-mypy in
/usr/local/google/home/ehudm/src/beam/build/gradleenv/1922375555/bin,
/usr/local/google/home/ehudm/src/beam/build/gradleenv/1922375555/bin,
/usr/local/google/home/ehudm/.pyenv/plugins/pyenv-virtualenv/shims,
/usr/local/google/home/ehudm/.pyenv/shims,
/usr/local/google/home/ehudm/.pyenv/bin, /usr/lib/google-golang/bin,
/usr/local/buildtools/java/jdk/bin, /usr/local/sbin, /usr/local/bin, /usr/sbin,
/usr/bin, /sbin, /bin, /google/data/ro/teams/iblaze,
/google/data/ro/projects/devtools/rebaser, /usr/local/google/home/ehudm/bin
```
Scan of subsequent run:
https://gradle.com/s/nobazytmko36c
It fails at the sdist task.
Perhaps your shell already has this binary installed, which is why it works
for you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 380254)
Time Spent: 3h 50m (was: 3h 40m)
> protoc-gen-mypy: program not found or is not executable
> -------------------------------------------------------
>
> Key: BEAM-8979
> URL: https://issues.apache.org/jira/browse/BEAM-8979
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
> Reporter: Kamil Wasilewski
> Assignee: Chad Dombrova
> Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> In some tests, `:sdks:python:sdist:` task fails due to problems in finding
> protoc-gen-mypy. The following tests are affected (there might be more):
> *
> [https://builds.apache.org/job/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/]
> *
> [https://builds.apache.org/job/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/
>
> |https://builds.apache.org/job/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/]
> Relevant logs:
> {code:java}
> 10:46:32 > Task :sdks:python:sdist FAILED
> 10:46:32 Requirement already satisfied: mypy-protobuf==1.12 in
> /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages
> (1.12)
> 10:46:32 beam_fn_api.proto: warning: Import google/protobuf/descriptor.proto
> but not used.
> 10:46:32 beam_fn_api.proto: warning: Import google/protobuf/wrappers.proto
> but not used.
> 10:46:32 protoc-gen-mypy: program not found or is not executable
> 10:46:32 --mypy_out: protoc-gen-mypy: Plugin failed with status code 1.
> 10:46:32
> /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages/setuptools/dist.py:476:
> UserWarning: Normalizing '2.19.0.dev' to '2.19.0.dev0'
> 10:46:32 normalized_version,
> 10:46:32 Traceback (most recent call last):
> 10:46:32 File "setup.py", line 295, in <module>
> 10:46:32 'mypy': generate_protos_first(mypy),
> 10:46:32 File
> "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages/setuptools/__init__.py",
> line 145, in setup
> 10:46:32 return distutils.core.setup(**attrs)
> 10:46:32 File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
> 10:46:32 dist.run_commands()
> 10:46:32 File "/usr/lib/python3.7/distutils/dist.py", line 966, in
> run_commands
> 10:46:32 self.run_command(cmd)
> 10:46:32 File "/usr/lib/python3.7/distutils/dist.py", line 985, in
> run_command
> 10:46:32 cmd_obj.run()
> 10:46:32 File
> "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages/setuptools/command/sdist.py",
> line 44, in run
> 10:46:32 self.run_command('egg_info')
> 10:46:32 File "/usr/lib/python3.7/distutils/cmd.py", line 313, in
> run_command
> 10:46:32 self.distribution.run_command(command)
> 10:46:32 File "/usr/lib/python3.7/distutils/dist.py", line 985, in
> run_command
> 10:46:32 cmd_obj.run()
> 10:46:32 File "setup.py", line 220, in run
> 10:46:32 gen_protos.generate_proto_files(log=log)
> 10:46:32 File
> "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/sdks/python/gen_protos.py",
> line 144, in generate_proto_files
> 10:46:32 '%s' % ret_code)
> 10:46:32 RuntimeError: Protoc returned non-zero status (see logs for
> details): 1
> {code}
>
> This is what I have tried so far to resolve this (without being successful):
> * Including _--plugin=protoc-gen-mypy=\{abs_path_to_executable}_ parameter
> to the _protoc_ call ingen_protos.py:131
> * Appending protoc-gen-mypy's directory to the PATH variable
> I wasn't able to reproduce this error locally.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
