No additional stacktraces. Full error output below.
It's not clear what is going wrong.
There isn't any exception from the subprocess execution since the
"WARNING:root:Installing grpcio-tools took 305.39 seconds." is printed.
Also, the time it takes to perform the install is equivalent to
successfully running the pip command.
I will report back if I find anything else. Currently doing the
explicit install via pip install -r sdks/python/build-requirements.txt
Thanks,
Thomas
WARNING:root:Installing grpcio-tools took 269.27 seconds.
INFO:gen_protos:Regenerating Python proto definitions (no output files).
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in
_bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File
"/src/streamingplatform/beam-release/beam/sdks/python/gen_protos.py", line
378, in _install_grpcio_tools_and_generate_proto_files
generate_proto_files(force=force)
File
"/src/streamingplatform/beam-release/beam/sdks/python/gen_protos.py", line
315, in generate_proto_files
protoc_gen_mypy = _find_protoc_gen_mypy()
File
"/src/streamingplatform/beam-release/beam/sdks/python/gen_protos.py", line
233, in _find_protoc_gen_mypy
(fname, ', '.join(search_paths)))
RuntimeError: Could not find protoc-gen-mypy in /code/venvs/venv2/bin,
/code/venvs/venv2/bin, /code/venvs/venv3/bin, /usr/local/sbin,
/usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin
Traceback (most recent call last):
File "setup.py", line 311, in <module>
'mypy': generate_protos_first(mypy),
File
"/code/venvs/venv2/local/lib/python2.7/site-packages/setuptools/__init__.py",
line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File
"/code/venvs/venv2/local/lib/python2.7/site-packages/wheel/bdist_wheel.py",
line 204, in run
self.run_command('build')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
self.run_command(cmd_name)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 235, in run
gen_protos.generate_proto_files()
File
"/src/streamingplatform/beam-release/beam/sdks/python/gen_protos.py", line
310, in generate_proto_files
raise ValueError("Proto generation failed (see log for details).")
ValueError: Proto generation failed (see log for details).
On Thu, May 7, 2020 at 2:25 PM Udi Meiri <[email protected]> wrote:
> It's hard to say without more details what's going on. Ahmet you're right
> that it installs build-requirements.txt and retries calling
> generate_proto_files().
>
> Thomas, were there additional stacktraces? (after a "During handling of
> the above exception, another exception occurred:" message?)
>
>
> On Thu, May 7, 2020 at 11:59 AM Ahmet Altay <[email protected]> wrote:
>
>>
>>
>> On Thu, May 7, 2020 at 11:56 AM Thomas Weise <[email protected]> wrote:
>>
>>> Thanks Udi! This is the issue. I'm trying to upgrade from 2.18 where
>>> build-requirements.txt didn't exist.
>>>
>>> Is there a reason why this cannot happen automatically when
>>> running python3.6 setup.py sdist bdist_wheel ?
>>>
>>
>> I _believe_ this should happen automatically here:
>> https://github.com/apache/beam/blob/master/sdks/python/gen_protos.py#L365.
>> Maybe there is a problem there?
>>
>>
>>>
>>> Thomas
>>>
>>>
>>> On Thu, May 7, 2020 at 11:07 AM Udi Meiri <[email protected]> wrote:
>>>
>>>> Probably not the issue, but double checking: are you running "pip
>>>> install -r sdks/python/build-requirements.txt" first?
>>>>
>>>> On Wed, May 6, 2020 at 7:22 PM Thomas Weise <[email protected]> wrote:
>>>>
>>>>> I'm working on rebasing our fork to 2.21.0 and run into a problem
>>>>> installing grpcio-tools that leads to *ModuleNotFoundError: No module
>>>>> named 'grpc_tools' *(see details below)
>>>>>
>>>>> I cannot reproduce this locally.
>>>>>
>>>>> Any =suggestions on what to look for?
>>>>>
>>>>> Thanks,
>>>>> Thomas
>>>>>
>>>>> [?25hBuilding wheels for collected packages: future, gr
>>>>> Running setup.py bdist_wheel for future ... [?25l- \
>>>>> [?25h Stored in directory:
>>>>> /root/.cache/pip/wheels/bf/c9/a3/c538d90ef17cf7823fa51fc701a7a7a910a80f6a405bf1
>>>>> Running setup.py bdist_wheel for grpcio ... [?25l- \ | / - \ | / - \
>>>>> | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ |
>>>>> /
>>>>> - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / -
>>>>> \
>>>>> | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ |
>>>>> /
>>>>> - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / -
>>>>> \
>>>>> | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ |
>>>>> /
>>>>> - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / -
>>>>> \
>>>>> | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ |
>>>>> /
>>>>> - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / -
>>>>> \
>>>>> | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ |
>>>>> /
>>>>> - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / -
>>>>> \
>>>>> | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ |
>>>>> /
>>>>> - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / -
>>>>> \
>>>>> | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ |
>>>>> /
>>>>> - \ | / - \ | / - \ |
>>>>> [?25h Stored in directory:
>>>>> /root/.cache/pip/wheels/00/4d/5f/07d0d4283911d2b917b867a11b1622d9d2cc8c286eefd1
>>>>> Successfully built future grpcio
>>>>> Installing collected packages: six, grpcio, setuptools, protobuf,
>>>>> grpcio-tools, future, mypy-protobuf
>>>>> Successfully installed future-0.16.0 grpcio-1.28.1 grpcio-tools-1.14.2
>>>>> mypy-protobuf-1.18 protobuf-3.11.3 setuptools-46.1.3 six-1.14.0
>>>>> WARNING:root:Installing grpcio-tools took 305.39 seconds.
>>>>> INFO:gen_protos:Regenerating Python proto definitions (no output
>>>>> files).
>>>>> Process Process-1:
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/src/streamingplatform/beam-release/beam/sdks/python/gen_protos.py", line
>>>>> 292, in generate_proto_files
>>>>> from grpc_tools import protoc
>>>>> ModuleNotFoundError: No module named 'grpc_tools'
>>>>>
>>>>> On Fri, Apr 10, 2020 at 10:01 AM Kyle Weaver <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> Just a heads up that the Beam 2.21 release branch [1] is cut.
>>>>>> - If you find any important issues that you think should be addressed
>>>>>> in the release, please tag the jira with fix version 2.21.0 and cc me
>>>>>> (username `ibzib`).
>>>>>> - Make sure to update the change log [2] with any significant changes
>>>>>> if you haven't already. Send a PR with the change and tag me. (I imagine
>>>>>> I'm not the only one who forgot to do this :).)
>>>>>>
>>>>>> Thanks,
>>>>>> Kyle
>>>>>>
>>>>>> [1] https://github.com/apache/beam/blob/release-2.21.0
>>>>>> [2] https://github.com/apache/beam/blob/master/CHANGES.md
>>>>>>
>>>>>
