See
<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/839/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9600] Get metrics in Flink uber jar job server.
[kcweaver] [BEAM-9600] Add a test for Flink REST metrics.
[rohde.samuel] [BEAM-9692] Replace the apply_read with a PTransformOverride
[bhulette] remove PipelineRunner.fromOptions call
[mxm] [BEAM-2128] Remove dependency on PipelineRunner#fromOptions to stage
[bhulette] Modify PipelineTest to check if exception is thrown on Pipeline run
[bhulette] Call FileSystems.setDefaultPipelineOptions in
[rohde.samuel] move redundant testing pieces to function
[mxm] [BEAM-9801] Pass in fire timestamp and pane info to timer callback
[mxm] [BEAM-9733] Repeatedly fire in batch mode until there are no more timers
[kcweaver] [BEAM-9860] Require job_endpoint when using PortableRunner.
[kcweaver] Replace portableWordCount tests with portableWordCountFlinkRunner.
[robertwb] Test simplification.
------------------------------------------
[...truncated 4.27 MB...]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
ERROR:root:java.lang.RuntimeException: Error received from SDK harness for
instruction 4: Traceback (most recent call last):
File "apache_beam/runners/worker/sdk_worker.py", line 245, in _execute
response = task()
File "apache_beam/runners/worker/sdk_worker.py", line 302, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "apache_beam/runners/worker/sdk_worker.py", line 471, in do_instruction
getattr(request, request_type), request.instruction_id)
File "apache_beam/runners/worker/sdk_worker.py", line 506, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "apache_beam/runners/worker/bundle_processor.py", line 913, in
process_bundle
element.data)
File "apache_beam/runners/worker/bundle_processor.py", line 217, in
process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 332, in output
cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 195, in receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 764, in process
delayed_application = self.dofn_runner.process_with_sized_restriction(o)
File "apache_beam/runners/common.py", line 975, in
process_with_sized_restriction
watermark_estimator=watermark_estimator)
File "apache_beam/runners/common.py", line 712, in invoke_process
windowed_value, additional_args, additional_kwargs)
File "apache_beam/runners/common.py", line 819, in _invoke_process_per_window
self.threadsafe_restriction_tracker.check_done()
File "apache_beam/runners/sdf_utils.py", line 115, in check_done
return self._restriction_tracker.check_done()
File "apache_beam/io/restriction_trackers.py", line 101, in check_done
self._range.stop))
ValueError: OffsetRestrictionTracker is not done since work in range [0, 6) has
not been claimed.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to FAILED
.sssINFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at
localhost:40569
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.22.0.dev.
If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.portable_runner:Waiting until the pipeline
has finished because the environment "LOOPBACK" has started a component
necessary for the execution.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
RUNNING
[flink-runner-job-invoker] WARN
org.apache.flink.runtime.webmonitor.WebMonitorUtils - Log file environment
variable 'log.file' is not set.
[flink-runner-job-invoker] WARN
org.apache.flink.runtime.webmonitor.WebMonitorUtils - JobManager log files are
unavailable in the web dashboard. Log file location not found in environment
variable 'log.file' or configuration key 'Key: 'web.log.path' , default: null
(fallback keys: [{key=jobmanager.web.log.path, isDeprecated=true}])'.
[[5]{Create, Map(<lambda at fn_runner_test.py:480>), WindowInto(WindowIntoFn),
Map(<lambda at fn_runner_test.py:483>)} (2/2)] WARN
org.apache.flink.metrics.MetricGroup - The operator name [5]{Create,
Map(<lambda at fn_runner_test.py:480>), WindowInto(WindowIntoFn), Map(<lambda
at fn_runner_test.py:483>)} exceeded the 80 characters length limit and was
truncated.
[[5]{Create, Map(<lambda at fn_runner_test.py:480>), WindowInto(WindowIntoFn),
Map(<lambda at fn_runner_test.py:483>)} (1/2)] WARN
org.apache.flink.metrics.MetricGroup - The operator name [5]{Create,
Map(<lambda at fn_runner_test.py:480>), WindowInto(WindowIntoFn), Map(<lambda
at fn_runner_test.py:483>)} exceeded the 80 characters length limit and was
truncated.
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel
for localhost:39585.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with
unbounded number of workers.
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for
localhost:34439.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data
plane.
Traceback (most recent call last):
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 689, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "DNS resolution failed"
debug_error_string =
"{"created":"@1588379350.529063335","description":"Failed to pick
subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3981,"referenced_errors":[{"created":"@1588379350.529057972","description":"Resolver
transient
failure","file":"src/core/ext/filters/client_channel/resolving_lb_policy.cc","file_line":214,"referenced_errors":[{"created":"@1588379350.529056060","description":"DNS
resolution
failed","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/dns_resolver_ares.cc","file_line":357,"grpc_status":14,"referenced_errors":[{"created":"@1588379350.529045364","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244,"referenced_errors":[{"created":"@1588379350.529011769","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244}]}]}]}]}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "apache_beam/runners/worker/data_plane.py", line 545, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 689, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "DNS resolution failed"
debug_error_string =
"{"created":"@1588379350.529063335","description":"Failed to pick
subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3981,"referenced_errors":[{"created":"@1588379350.529057972","description":"Resolver
transient
failure","file":"src/core/ext/filters/client_channel/resolving_lb_policy.cc","file_line":214,"referenced_errors":[{"created":"@1588379350.529056060","description":"DNS
resolution
failed","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/dns_resolver_ares.cc","file_line":357,"grpc_status":14,"referenced_errors":[{"created":"@1588379350.529045364","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244,"referenced_errors":[{"created":"@1588379350.529011769","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244}]}]}]}]}"
>
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for
localhost:34895
[assert_that/Group/GroupByKey -> [3]assert_that/{Group, Unkey, Match} (1/2)]
WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for
unknown endpoint.
INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight
requests to complete
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data
plane.
Traceback (most recent call last):
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string =
"{"created":"@1588379351.141953580","description":"Error received from peer
ipv4:127.0.0.1:34895","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Multiplexer
hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "apache_beam/runners/worker/data_plane.py", line 545, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string =
"{"created":"@1588379351.141953580","description":"Error received from peer
ipv4:127.0.0.1:34895","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Multiplexer
hanging up","grpc_status":1}"
>
INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data
channels.
INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state
handlers.
INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
.INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at
localhost:41901
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.22.0.dev.
If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.portable_runner:Waiting until the pipeline
has finished because the environment "LOOPBACK" has started a component
necessary for the execution.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
RUNNING
[flink-runner-job-invoker] WARN
org.apache.flink.runtime.webmonitor.WebMonitorUtils - Log file environment
variable 'log.file' is not set.
[flink-runner-job-invoker] WARN
org.apache.flink.runtime.webmonitor.WebMonitorUtils - JobManager log files are
unavailable in the web dashboard. Log file location not found in environment
variable 'log.file' or configuration key 'Key: 'web.log.path' , default: null
(fallback keys: [{key=jobmanager.web.log.path, isDeprecated=true}])'.
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel
for localhost:33795.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with
unbounded number of workers.
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for
localhost:46703.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data
plane.
Traceback (most recent call last):
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 689, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "DNS resolution failed"
debug_error_string =
"{"created":"@1588379352.318124382","description":"Failed to pick
subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3981,"referenced_errors":[{"created":"@1588379352.318120756","description":"Resolver
transient
failure","file":"src/core/ext/filters/client_channel/resolving_lb_policy.cc","file_line":214,"referenced_errors":[{"created":"@1588379352.318119520","description":"DNS
resolution
failed","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/dns_resolver_ares.cc","file_line":357,"grpc_status":14,"referenced_errors":[{"created":"@1588379352.318111677","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244,"referenced_errors":[{"created":"@1588379352.318066943","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244}]}]}]}]}"
>
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for
localhost:34041
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "apache_beam/runners/worker/data_plane.py", line 545, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 689, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "DNS resolution failed"
debug_error_string =
"{"created":"@1588379352.318124382","description":"Failed to pick
subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3981,"referenced_errors":[{"created":"@1588379352.318120756","description":"Resolver
transient
failure","file":"src/core/ext/filters/client_channel/resolving_lb_policy.cc","file_line":214,"referenced_errors":[{"created":"@1588379352.318119520","description":"DNS
resolution
failed","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/dns_resolver_ares.cc","file_line":357,"grpc_status":14,"referenced_errors":[{"created":"@1588379352.318111677","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244,"referenced_errors":[{"created":"@1588379352.318066943","description":"C-ares
status is not ARES_SUCCESS: Misformatted domain
name","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244}]}]}]}]}"
>
[assert_that/Group/GroupByKey -> [3]assert_that/{Group, Unkey, Match} (1/2)]
WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for
unknown endpoint.
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data
plane.
Traceback (most recent call last):
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string =
"{"created":"@1588379352.696651548","description":"Error received from peer
ipv4:127.0.0.1:34041","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Multiplexer
hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "apache_beam/runners/worker/data_plane.py", line 545, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "apache_beam/runners/worker/data_plane.py", line 528, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 413, in next
return self._next()
File
"<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string =
"{"created":"@1588379352.696651548","description":"Error received from peer
ipv4:127.0.0.1:34041","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Multiplexer
hanging up","grpc_status":1}"
>
INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight
requests to complete
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
.INFO:__main__:removing conf dir: /tmp/flinktest-confzqgprq
INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data
channels.
INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state
handlers.
INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
----------------------------------------------------------------------
Ran 80 tests in 148.489s
OK (skipped=16)
FAILURE: Build failed with an exception.
* Where:
Script
'<https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/common.gradle'>
line: 55
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py2:flinkCompatibilityMatrixBatchLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 38s
76 actionable tasks: 57 executed, 18 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/n3unixm4q4zkq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
