[ https://issues.apache.org/jira/browse/BEAM-8979?focusedWorklogId=382110&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382110 ]
ASF GitHub Bot logged work on BEAM-8979: ---------------------------------------- Author: ASF GitHub Bot Created on: 05/Feb/20 07:08 Start Date: 05/Feb/20 07:08 Worklog Time Spent: 10m Work Description: chadrik commented on issue #10734: [BEAM-8979] reintroduce mypy-protobuf stub generation URL: https://github.com/apache/beam/pull/10734#issuecomment-582271275 Here are the results of my investigation tonight. I cloned mypy-protobuf and created a virtualenv with a path that exceeded the 512 character limit on darwin. I tried installing the scripts in several ways: | command | setup.py uses entry_points | package format | produces #!/bin/sh | | - | - | - |- | | `pip install python/` | no | wheel | no | | `cd python/ && python setup.py install` | no | egg | no | | `pip install python/` | yes | wheel | yes | | `cd python/ && python setup.py install` | yes | egg | no | | `pip wheel -w . python/ && pip install mypy_protobuf-1.17-py2-none-any.whl` | yes | wheel | yes | This confirms that you need to use pip to install (which we are), and entry_points in the project's setup.py file, for the fix to be applied to the installed script. All tests were performed with the following: - python 2.7.16 - pip 20.0.2 My changes are here: https://github.com/chadrik/mypy-protobuf/tree/setup_entry_points Next I'm going to try editing the build-requirements.txt to point at my github fork to see if we can get a working test. If that works, then I'll make the upstream PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 382110) Time Spent: 6h 40m (was: 6.5h) > protoc-gen-mypy: program not found or is not executable > ------------------------------------------------------- > > Key: BEAM-8979 > URL: https://issues.apache.org/jira/browse/BEAM-8979 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, test-failures > Reporter: Kamil Wasilewski > Assignee: Chad Dombrova > Priority: Major > Time Spent: 6h 40m > Remaining Estimate: 0h > > In some tests, `:sdks:python:sdist:` task fails due to problems in finding > protoc-gen-mypy. The following tests are affected (there might be more): > * > [https://builds.apache.org/job/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/] > * > [https://builds.apache.org/job/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/ > > |https://builds.apache.org/job/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/] > Relevant logs: > {code:java} > 10:46:32 > Task :sdks:python:sdist FAILED > 10:46:32 Requirement already satisfied: mypy-protobuf==1.12 in > /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages > (1.12) > 10:46:32 beam_fn_api.proto: warning: Import google/protobuf/descriptor.proto > but not used. > 10:46:32 beam_fn_api.proto: warning: Import google/protobuf/wrappers.proto > but not used. > 10:46:32 protoc-gen-mypy: program not found or is not executable > 10:46:32 --mypy_out: protoc-gen-mypy: Plugin failed with status code 1. > 10:46:32 > /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages/setuptools/dist.py:476: > UserWarning: Normalizing '2.19.0.dev' to '2.19.0.dev0' > 10:46:32 normalized_version, > 10:46:32 Traceback (most recent call last): > 10:46:32 File "setup.py", line 295, in <module> > 10:46:32 'mypy': generate_protos_first(mypy), > 10:46:32 File > "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages/setuptools/__init__.py", > line 145, in setup > 10:46:32 return distutils.core.setup(**attrs) > 10:46:32 File "/usr/lib/python3.7/distutils/core.py", line 148, in setup > 10:46:32 dist.run_commands() > 10:46:32 File "/usr/lib/python3.7/distutils/dist.py", line 966, in > run_commands > 10:46:32 self.run_command(cmd) > 10:46:32 File "/usr/lib/python3.7/distutils/dist.py", line 985, in > run_command > 10:46:32 cmd_obj.run() > 10:46:32 File > "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/build/gradleenv/1922375555/lib/python3.7/site-packages/setuptools/command/sdist.py", > line 44, in run > 10:46:32 self.run_command('egg_info') > 10:46:32 File "/usr/lib/python3.7/distutils/cmd.py", line 313, in > run_command > 10:46:32 self.distribution.run_command(command) > 10:46:32 File "/usr/lib/python3.7/distutils/dist.py", line 985, in > run_command > 10:46:32 cmd_obj.run() > 10:46:32 File "setup.py", line 220, in run > 10:46:32 gen_protos.generate_proto_files(log=log) > 10:46:32 File > "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/src/sdks/python/gen_protos.py", > line 144, in generate_proto_files > 10:46:32 '%s' % ret_code) > 10:46:32 RuntimeError: Protoc returned non-zero status (see logs for > details): 1 > {code} > > This is what I have tried so far to resolve this (without being successful): > * Including _--plugin=protoc-gen-mypy=\{abs_path_to_executable}_ parameter > to the _protoc_ call ingen_protos.py:131 > * Appending protoc-gen-mypy's directory to the PATH variable > I wasn't able to reproduce this error locally. > -- This message was sent by Atlassian Jira (v8.3.4#803005)