[
https://issues.apache.org/jira/browse/BEAM-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017732#comment-17017732
]
Chad Dombrova commented on BEAM-9130:
-------------------------------------
I updated my PR. There are still other problems with it, but I solved the
google.protobuf import problem.
There are a few pieces to the solution:
* use a pyproject.toml file to specify build requirements:
[https://github.com/apache/beam/pull/10038/files#diff-71ede7e13a310beaedc4466b48096e26R18]
* use pep517 module to build source:
[https://github.com/apache/beam/pull/10038/files#diff-d3cc66016c37760e131f228abbee6667R34]
* add pep517's temp build directory to the site dir to allow the google
package's .pth file to work:
[https://github.com/apache/beam/pull/10038/commits/ed06bf1b17e5f60d7f6bcf7766b93e4eaf4b86ff]
I'm sure there are easier ways to hack this to work without such dramatic
changes, but this approach sets us on a course to untangle the mess that we
have now.
> sdks:python:test-suites:direct:py2:hdfsIntegrationTest is failing with
> ImportError: No module named google.protobuf.message
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-9130
> URL: https://issues.apache.org/jira/browse/BEAM-9130
> Project: Beam
> Issue Type: Improvement
> Components: test-failures
> Reporter: Valentyn Tymofieiev
> Priority: Major
> Labels: currently-failing
>
> From logs:
> {noformat}
> 16:33:50 File "/usr/local/lib/python2.7/multiprocessing/process.py", line
> 267, in _bootstrap
> 16:33:50 [0m[91m self.run()
> 16:33:50 File "/usr/local/lib/python2.7/multiprocessing/process.py", line
> 114, in run
> 16:33:50 self._target(*self._args, **self._kwargs)
> 16:33:50 File "/app/sdks/python/gen_protos.py", line 357, in
> _install_grpcio_tools_and_generate_proto_files
> 16:33:50 generate_proto_files(force=force)
> 16:33:50 File "/app/sdks/python/gen_protos.py", line 324, in
> generate_proto_files
> 16:33:50 generate_urn_files(log, out_dir)
> 16:33:50 File "/app/sdks/python/gen_protos.py", line 65, in
> generate_urn_files
> 16:33:50 import google.protobuf.message as message
> 16:33:50 [0m[91mImportError: No module named google.protobuf.message
> 16:33:50 [0m[91mTraceback (most recent call last):
> 16:33:50 File "setup.py", line 305, in <module>
> 16:33:50 'mypy': generate_protos_first(mypy),
> 16:33:50 File
> "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 145, in
> setup
> 16:33:50 [0m[91m return distutils.core.setup(**attrs)
> 16:33:50 File "/usr/local/lib/python2.7/distutils/core.py", line 151, in
> setup
> 16:33:50 [0m[91m dist.run_commands()
> 16:33:50 File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in
> run_commands
> 16:33:50 [0m[91m self.run_command(cmd)
> 16:33:50 File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in
> run_command
> 16:33:50 cmd_obj.run()
> 16:33:50 File
> "/usr/local/lib/python2.7/site-packages/setuptools/command/sdist.py", line
> 44, in run
> 16:33:50 [0m[91m self.run_command('egg_info')
> 16:33:50 File "/usr/local/lib/python2.7/distutils/cmd.py", line 326, in
> run_command
> 16:33:50 [0m[91m self.distribution.run_command(command)
> 16:33:50 File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in
> run_command
> 16:33:50 [0m[91m cmd_obj.run()
> 16:33:50 File "setup.py", line 229, in run
> 16:33:50 [0m[91m gen_protos.generate_proto_files(log=log)
> 16:33:50 File "/app/sdks/python/gen_protos.py", line 291, in
> generate_proto_files
> 16:33:50 raise ValueError("Proto generation failed (see log for
> details).")
> 16:33:50 [0m[91mValueError: [0m[91mProto generation failed (see log for
> details
> {noformat}
> {noformat}
> import google.protobuf.message as message
> ImportError: No module named google.protobuf.message
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
