[
https://issues.apache.org/jira/browse/BEAM-7993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917275#comment-16917275
]
Kyle Weaver edited comment on BEAM-7993 at 8/27/19 11:50 PM:
-------------------------------------------------------------
[https://builds.apache.org/job/beam_PreCommit_Portable_Python_Commit/5720/consoleFull]
We now have container logs for the most recent failure. Looks like this is the
root cause:
16:04:21 Traceback (most recent call last):
16:04:21 File "/usr/local/lib/python3.5/runpy.py", line 183, in
_run_module_as_main
16:04:21 mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
16:04:21 File "/usr/local/lib/python3.5/runpy.py", line 109, in
_get_module_details
16:04:21 __import__(pkg_name)
16:04:21 File "/usr/local/lib/python3.5/site-packages/apache_beam/__init__.py",
line 97, in <module>
16:04:21 from apache_beam import coders
16:04:21 File
"/usr/local/lib/python3.5/site-packages/apache_beam/coders/__init__.py", line
19, in <module>
16:04:21 from apache_beam.coders.coders import *
16:04:21 File
"/usr/local/lib/python3.5/site-packages/apache_beam/coders/coders.py", line 33,
in <module>
16:04:21 from apache_beam.coders import coder_impl
16:04:21 File "apache_beam/utils/windowed_value.pxd", line 28, in init
apache_beam.coders.coder_impl
16:04:21 File "apache_beam/utils/windowed_value.py", line 34, in init
apache_beam.utils.windowed_value
16:04:21 File
"/usr/local/lib/python3.5/site-packages/apache_beam/utils/timestamp.py", line
34, in <module>
16:04:21 from apache_beam.portability import common_urns
16:04:21 File
"/usr/local/lib/python3.5/site-packages/apache_beam/portability/common_urns.py",
line 24, in <module>
16:04:21 from apache_beam.portability.api import beam_runner_api_pb2
16:04:21 File
"/usr/local/lib/python3.5/site-packages/apache_beam/portability/api/beam_runner_api_pb2.py",
line 16, in <module>
16:04:21 import endpoints_pb2 as endpoints__pb2
16:04:21 ImportError: No module named 'endpoints_pb2'
was (Author: ibzib):
[https://builds.apache.org/job/beam_PreCommit_Portable_Python_Commit/5720/consoleFull]
We now have container logs for the most recent failure. Looks like this is the
root cause:
*16:04:21* Traceback (most recent call last):*16:04:21* File
"/usr/local/lib/python3.5/runpy.py", line 183, in _run_module_as_main*16:04:21*
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)*16:04:21*
File "/usr/local/lib/python3.5/runpy.py", line 109, in
_get_module_details*16:04:21* __import__(pkg_name)*16:04:21* File
"/usr/local/lib/python3.5/site-packages/apache_beam/__init__.py", line 97, in
<module>*16:04:21* from apache_beam import coders*16:04:21* File
"/usr/local/lib/python3.5/site-packages/apache_beam/coders/__init__.py", line
19, in <module>*16:04:21* from apache_beam.coders.coders import **16:04:21*
File "/usr/local/lib/python3.5/site-packages/apache_beam/coders/coders.py",
line 33, in <module>*16:04:21* from apache_beam.coders import
coder_impl*16:04:21* File "apache_beam/utils/windowed_value.pxd", line 28, in
init apache_beam.coders.coder_impl*16:04:21* File
"apache_beam/utils/windowed_value.py", line 34, in init
apache_beam.utils.windowed_value*16:04:21* File
"/usr/local/lib/python3.5/site-packages/apache_beam/utils/timestamp.py", line
34, in <module>*16:04:21* from apache_beam.portability import
common_urns*16:04:21* File
"/usr/local/lib/python3.5/site-packages/apache_beam/portability/common_urns.py",
line 24, in <module>*16:04:21* from apache_beam.portability.api import
beam_runner_api_pb2*16:04:21* File
"/usr/local/lib/python3.5/site-packages/apache_beam/portability/api/beam_runner_api_pb2.py",
line 16, in <module>*16:04:21* import endpoints_pb2 as
endpoints__pb2*16:04:21* ImportError: No module named 'endpoints_pb2'
> portable python precommit is flaky
> ----------------------------------
>
> Key: BEAM-7993
> URL: https://issues.apache.org/jira/browse/BEAM-7993
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures, testing
> Affects Versions: 2.15.0
> Reporter: Udi Meiri
> Assignee: Kyle Weaver
> Priority: Major
> Labels: currently-failing
> Fix For: 2.16.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I'm not sure what the root cause is here.
> Example log where
> :sdks:python:test-suites:portable:py35:portableWordCountBatch failed:
> {code}
> 11:51:22 [CHAIN MapPartition (MapPartition at [1]read/Read/Split) -> FlatMap
> (FlatMap at ExtractOutput[0]) (2/2)] ERROR
> org.apache.flink.runtime.operators.BatchTask - Error in task code: CHAIN
> MapPartition (MapPartition at [1]read/Read/Split) -> FlatMap (FlatMap at
> ExtractOutput[0]) (2/2)
> 11:51:22 [CHAIN MapPartition (MapPartition at [1]read/Read/Split) -> FlatMap
> (FlatMap at ExtractOutput[0]) (1/2)] ERROR
> org.apache.flink.runtime.operators.BatchTask - Error in task code: CHAIN
> MapPartition (MapPartition at [1]read/Read/Split) -> FlatMap (FlatMap at
> ExtractOutput[0]) (1/2)
> 11:51:22 [CHAIN MapPartition (MapPartition at
> [2]write/Write/WriteImpl/DoOnce/{FlatMap(<lambda at core.py:2457>),
> Map(decode)}) -> FlatMap (FlatMap at ExtractOutput[0]) (2/2)] ERROR
> org.apache.flink.runtime.operators.BatchTask - Error in task code: CHAIN
> MapPartition (MapPartition at
> [2]write/Write/WriteImpl/DoOnce/{FlatMap(<lambda at core.py:2457>),
> Map(decode)}) -> FlatMap (FlatMap at ExtractOutput[0]) (2/2)
> 11:51:22 [CHAIN MapPartition (MapPartition at
> [2]write/Write/WriteImpl/DoOnce/{FlatMap(<lambda at core.py:2457>),
> Map(decode)}) -> FlatMap (FlatMap at ExtractOutput[0]) (1/2)] ERROR
> org.apache.flink.runtime.operators.BatchTask - Error in task code: CHAIN
> MapPartition (MapPartition at
> [2]write/Write/WriteImpl/DoOnce/{FlatMap(<lambda at core.py:2457>),
> Map(decode)}) -> FlatMap (FlatMap at ExtractOutput[0]) (1/2)
> 11:51:22 java.lang.Exception: The user defined 'open()' method caused an
> exception: java.io.IOException: Received exit code 1 for command 'docker
> inspect -f {{.State.Running}}
> 642c312c335d3881b885873c66917b536e79cff07503fdceaddee5fbeb10bfd1'. stderr:
> Error: No such object:
> 642c312c335d3881b885873c66917b536e79cff07503fdceaddee5fbeb10bfd1
> 11:51:22 at
> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498)
> 11:51:22 at
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
> 11:51:22 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
> 11:51:22 at java.lang.Thread.run(Thread.java:748)
> 11:51:22 Caused by:
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> java.io.IOException: Received exit code 1 for command 'docker inspect -f
> {{.State.Running}}
> 642c312c335d3881b885873c66917b536e79cff07503fdceaddee5fbeb10bfd1'. stderr:
> Error: No such object:
> 642c312c335d3881b885873c66917b536e79cff07503fdceaddee5fbeb10bfd1
> 11:51:22 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
> 11:51:22 at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:211)
> 11:51:22 at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:202)
> 11:51:22 at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:185)
> 11:51:22 at
> org.apache.beam.runners.flink.translation.functions.FlinkDefaultExecutableStageContext.getStageBundleFactory(FlinkDefaultExecutableStageContext.java:49)
> 11:51:22 at
> org.apache.beam.runners.flink.translation.functions.ReferenceCountingFlinkExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingFlinkExecutableStageContextFactory.java:203)
> 11:51:22 at
> org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.open(FlinkExecutableStageFunction.java:129)
> 11:51:22 at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> 11:51:22 at
> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:494)
> 11:51:22 ... 3 more
> {code}
> https://builds.apache.org/job/beam_PreCommit_Portable_Python_Commit/5512/consoleFull
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
