See
<https://builds.apache.org/job/beam_PreCommit_Python_Cron/2111/display/redirect?page=changes>
Changes:
[chadrik] [BEAM-8523] JobAPI: Give access to timestamped state change history
[chadrik] Rename GetJobStateResponse to JobStateEvent
[chadrik] Move state history utilities to AbstractBeamJob
[chadrik] Small bugfix to FlinkBeamJob job state mapping
[chadrik] Fix existing bugs in AbstractJobServiceServicer
[chadrik] Use timestamp.Timestamp instead of float
[echauchot] [BEAM-8470] Update capability matrix: add Spark Structured Streaming
[echauchot] [BEAM-8470] Update Spark runner page: add Spark Structured Streaming
------------------------------------------
[...truncated 6.10 MB...]
self._pipeline_proto)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 399, in run_via_runner_api
return self.run_stages(stage_context, stages)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 481, in run_stages
stage_context.safe_coders)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 777, in _run_stage
result, splits = bundle_manager.process_bundle(data_input, data_output)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1814, in process_bundle
part, expected_outputs), part_inputs):
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/target/.tox-py27-cython/py27-cython/local/lib/python2.7/site-packages/concurrent/futures/_base.py";,>
line 641, in result_iterator
yield fs.pop().result()
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/target/.tox-py27-cython/py27-cython/local/lib/python2.7/site-packages/concurrent/futures/_base.py";,>
line 462, in result
return self.__get_result()
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/utils/thread_pool_executor.py";,>
line 42, in run
self._future.set_result(self._fn(*self._fn_args, **self._fn_kwargs))
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1814, in <lambda>
part, expected_outputs), part_inputs):
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1750, in process_bundle
result_future = self._worker_handler.control_conn.push(process_bundle_req)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1176, in push
response = self.worker.do_instruction(request)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py";,>
line 284, in do_instruction
request.instruction_id)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py";,>
line 310, in process_bundle
bundle_processor.process_bundle(instruction_id))
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py";,>
line 675, in process_bundle
data.transform_id].process_encoded(data.data)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py";,>
line 146, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 258, in
apache_beam.runners.worker.operations.Operation.output
def output(self, windowed_value, output_index=0):
File "apache_beam/runners/worker/operations.py", line 259, in
apache_beam.runners.worker.operations.Operation.output
cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 146, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 596, in
apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 597, in
apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 808, in
apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 814, in
apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 866, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 812, in
apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 447, in
apache_beam.runners.common.SimpleInvoker.invoke_process
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 951, in
apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 146, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 596, in
apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 597, in
apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 808, in
apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 814, in
apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 866, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 812, in
apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 447, in
apache_beam.runners.common.SimpleInvoker.invoke_process
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 951, in
apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 146, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 596, in
apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 597, in
apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 808, in
apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 814, in
apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 881, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
raise_with_traceback(new_exn)
File "apache_beam/runners/common.py", line 812, in
apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 448, in
apache_beam.runners.common.SimpleInvoker.invoke_process
windowed_value, self.process_method(windowed_value.value))
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/transforms/core.py";,>
line 1416, in <lambda>
wrapper = lambda x: [fn(x)]
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py";,>
line 630, in first
return second(x)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py";,>
line 633, in second
return third(x)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py";,>
line 636, in third
raise ValueError('x')
ValueError: x [while running 'Map(first)']
apache_beam.runners.portability.local_job_service: ERROR: <module 'traceback'
from '/usr/lib/python2.7/traceback.pyc'>
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/local_job_service.py";,>
line 232, in _run_job
self._pipeline_proto)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 399, in run_via_runner_api
return self.run_stages(stage_context, stages)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 481, in run_stages
stage_context.safe_coders)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 777, in _run_stage
result, splits = bundle_manager.process_bundle(data_input, data_output)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1814, in process_bundle
part, expected_outputs), part_inputs):
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/target/.tox-py27-cython/py27-cython/local/lib/python2.7/site-packages/concurrent/futures/_base.py";,>
line 641, in result_iterator
yield fs.pop().result()
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/target/.tox-py27-cython/py27-cython/local/lib/python2.7/site-packages/concurrent/futures/_base.py";,>
line 462, in result
return self.__get_result()
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/utils/thread_pool_executor.py";,>
line 42, in run
self._future.set_result(self._fn(*self._fn_args, **self._fn_kwargs))
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1814, in <lambda>
part, expected_outputs), part_inputs):
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1750, in process_bundle
result_future = self._worker_handler.control_conn.push(process_bundle_req)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py";,>
line 1176, in push
response = self.worker.do_instruction(request)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py";,>
line 284, in do_instruction
request.instruction_id)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py";,>
line 310, in process_bundle
bundle_processor.process_bundle(instruction_id))
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py";,>
line 675, in process_bundle
data.transform_id].process_encoded(data.data)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py";,>
line 146, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 258, in
apache_beam.runners.worker.operations.Operation.output
def output(self, windowed_value, output_index=0):
File "apache_beam/runners/worker/operations.py", line 259, in
apache_beam.runners.worker.operations.Operation.output
cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 146, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 596, in
apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 597, in
apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 808, in
apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 814, in
apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 866, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 812, in
apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 447, in
apache_beam.runners.common.SimpleInvoker.invoke_process
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 951, in
apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 146, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 596, in
apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 597, in
apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 808, in
apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 814, in
apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 866, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 812, in
apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 447, in
apache_beam.runners.common.SimpleInvoker.invoke_process
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 951, in
apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 146, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 596, in
apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 597, in
apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 808, in
apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 814, in
apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 881, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
raise_with_traceback(new_exn)
File "apache_beam/runners/common.py", line 812, in
apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 448, in
apache_beam.runners.common.SimpleInvoker.invoke_process
windowed_value, self.process_method(windowed_value.value))
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/transforms/core.py";,>
line 1416, in <lambda>
wrapper = lambda x: [fn(x)]
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py";,>
line 630, in first
return second(x)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py";,>
line 633, in second
return third(x)
File
"<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py";,>
line 636, in third
raise ValueError('x')
ValueError: x [while running 'Map(first)']
apache_beam.runners.portability.portable_runner: INFO: Job state changed to
STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to
STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to
RUNNING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to
FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 2958 tests in 2671.165s
FAILED (SKIP=562, failures=1)
ERROR: InvocationError for command
<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/target/.tox-py27-cython/py27-cython/bin/python>
setup.py nosetests --ignore-files '.*py3\d?\.py$' (exited with code 1)
py27-cython run-test-post: commands[0] |
<https://builds.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
___________________________________ summary ____________________________________
ERROR: py27-cython: commands failed
> Task :sdks:python:test-suites:tox:py2:testPy2Cython FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py2:testPy2Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 35m 54s
78 actionable tasks: 61 executed, 17 from cache
Publishing build scan...
Publishing failed.
The response from https://scans-in.gradle.com/in/5.2.1/2.3 was not from the
build scan server.
Your network environment may be interfering, or the service may be unavailable.
If you believe this to be in error, please report this problem via
https://gradle.com/scans/help/plugin and include the following via copy/paste:
----------
Gradle version: 5.2.1
Plugin version: 2.3
Request URL: https://scans-in.gradle.com/in/5.2.1/2.3
Request ID: 73967a72-0d50-413d-9856-4d9a00c15066
Response status code: 502
Response content type: text/html; charset=UTF-8
Response server type: cloudflare
----------
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org
For additional commands, e-mail: builds-h...@beam.apache.org
