See
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/1027/display/redirect?page=changes>
Changes:
[pabloem] Beam Katas - Update Beam to v2.12.0, javadoc, pydoc to use 'current'
------------------------------------------
[...truncated 956.90 KB...]
File "apache_beam/runners/common.py", line 666, in
apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_file_loads.py",
line 441, in process
'BigQuery jobs failed. BQ error: %s', self._latest_error)
Exception: ('BigQuery jobs failed. BQ error: %s', <JobStatus
errorResult: <ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>
errors: [<ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>]
state: 'DONE'>)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py",
line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 176, in execute
op.start()
File "dataflow_worker/native_operations.py", line 38, in
dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 39, in
dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 44, in
dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 54, in
dataflow_worker.native_operations.NativeReadOperation.start
File "apache_beam/runners/worker/operations.py", line 256, in
apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 143, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 593, in
apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 594, in
apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 778, in
apache_beam.runners.common.DoFnRunner.receive
File "apache_beam/runners/common.py", line 784, in
apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
File "/usr/local/lib/python3.7/site-packages/future/utils/__init__.py", line
421, in raise_with_traceback
raise exc.with_traceback(traceback)
File "apache_beam/runners/common.py", line 782, in
apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 594, in
apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 666, in
apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_file_loads.py",
line 441, in process
'BigQuery jobs failed. BQ error: %s', self._latest_error)
Exception: ("BigQuery jobs failed. BQ error: %s [while running
'WriteWithMultipleDests/BigQueryBatchFileLoads/WaitForCopyJobs/WaitForCopyJobs']",
<JobStatus
errorResult: <ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>
errors: [<ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>]
state: 'DONE'>)
root: INFO: 2019-06-03T17:58:57.217Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "apache_beam/runners/common.py", line 782, in
apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 594, in
apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 666, in
apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_file_loads.py",
line 441, in process
'BigQuery jobs failed. BQ error: %s', self._latest_error)
Exception: ('BigQuery jobs failed. BQ error: %s', <JobStatus
errorResult: <ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>
errors: [<ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>]
state: 'DONE'>)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py",
line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 176, in execute
op.start()
File "dataflow_worker/native_operations.py", line 38, in
dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 39, in
dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 44, in
dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 54, in
dataflow_worker.native_operations.NativeReadOperation.start
File "apache_beam/runners/worker/operations.py", line 256, in
apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 143, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 593, in
apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 594, in
apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 778, in
apache_beam.runners.common.DoFnRunner.receive
File "apache_beam/runners/common.py", line 784, in
apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in
apache_beam.runners.common.DoFnRunner._reraise_augmented
File "/usr/local/lib/python3.7/site-packages/future/utils/__init__.py", line
421, in raise_with_traceback
raise exc.with_traceback(traceback)
File "apache_beam/runners/common.py", line 782, in
apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 594, in
apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 666, in
apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_file_loads.py",
line 441, in process
'BigQuery jobs failed. BQ error: %s', self._latest_error)
Exception: ("BigQuery jobs failed. BQ error: %s [while running
'WriteWithMultipleDests/BigQueryBatchFileLoads/WaitForCopyJobs/WaitForCopyJobs']",
<JobStatus
errorResult: <ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>
errors: [<ErrorProto
message: 'Not found: Table
apache-beam-testing:python_bq_file_loads_15595843114095.beam_load_2019_06_03_175817_9_5b3b6196cef9fd926078ec31862c8d0b_0
was not found in location US'
reason: 'notFound'>]
state: 'DONE'>)
root: INFO: 2019-06-03T17:58:57.284Z: JOB_MESSAGE_DEBUG: Executing failure step
failure91
root: INFO: 2019-06-03T17:58:57.331Z: JOB_MESSAGE_ERROR: Workflow failed.
Causes:
S30:WriteWithMultipleDests/BigQueryBatchFileLoads/ImpulseMonitorCopyJobs/Read+WriteWithMultipleDests/BigQueryBatchFileLoads/WaitForCopyJobs/WaitForCopyJobs+WriteWithMultipleDests/BigQueryBatchFileLoads/RemoveTempTables/PassTables/PassTables+WriteWithMultipleDests/BigQueryBatchFileLoads/RemoveTempTables/AddUselessValue+WriteWithMultipleDests/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Reify+WriteWithMultipleDests/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Write
failed., The job failed because a work item has failed 4 times. Look in
previous log entries for the cause of each one of the 4 failures. For more
information, see https://cloud.google.com/dataflow/docs/guides/common-errors.
The work item was attempted on these workers:
beamapp-jenkins-060317515-06031052-vg9u-harness-pr2g,
beamapp-jenkins-060317515-06031052-vg9u-harness-pr2g,
beamapp-jenkins-060317515-06031052-vg9u-harness-pr2g,
beamapp-jenkins-060317515-06031052-vg9u-harness-pr2g
root: INFO: 2019-06-03T17:58:57.476Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2019-06-03T17:58:57.543Z: JOB_MESSAGE_DEBUG: Starting worker pool
teardown.
root: INFO: 2019-06-03T17:58:57.588Z: JOB_MESSAGE_BASIC: Stopping worker pool...
root: INFO: 2019-06-03T18:01:42.789Z: JOB_MESSAGE_DETAILED: Autoscaling:
Reduced the number of workers to 0 based on the rate of progress in the
currently running step(s).
root: INFO: 2019-06-03T18:01:42.917Z: JOB_MESSAGE_BASIC: Worker pool stopped.
root: INFO: 2019-06-03T18:01:43.042Z: JOB_MESSAGE_DEBUG: Tearing down pending
resources...
root: INFO: Job 2019-06-03_10_52_01-15275042995928089772 is in state
JOB_STATE_FAILED
root: INFO: Deleting dataset python_bq_file_loads_15595843114095 in project
apache-beam-testing
--------------------- >> end captured logging << ---------------------
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_35-2374490517156578401?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1137:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
method_to_use = self._compute_method(p, p.options)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_58_27-5848347732925691594?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_06_35-17582001628648852314?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_13_56-153302433796792396?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_26-3313410178014083378?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:680:
BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use
WriteToBigQuery instead.
kms_key=transform.kms_key))
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_02_29-5044199623626743355?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_10_55-5868530627665457752?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_29-17186740128152381527?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_54_26-5786930508189509176?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1137:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
method_to_use = self._compute_method(p, p.options)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_02_29-11959315163732711271?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_11_30-9247492764963099961?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/fileio_test.py>:218:
FutureWarning: MatchAll is experimental.
| 'GetPath' >> beam.Map(lambda metadata: metadata.path))
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_19_51-418087710397727115?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/fileio_test.py>:229:
FutureWarning: MatchAll is experimental.
| 'Checksums' >> beam.Map(compute_hash))
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/fileio_test.py>:229:
FutureWarning: ReadMatches is experimental.
| 'Checksums' >> beam.Map(compute_hash))
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_28-6042583757470503405?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_49_39-12630259763641864582?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1137:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
method_to_use = self._compute_method(p, p.options)
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:545:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
or p.options.view_as(GoogleCloudOptions).temp_location)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_58_38-4930457403763491845?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_06_03-17562885896846634613?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_27-8141563563975566132?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:680:
BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use
WriteToBigQuery instead.
kms_key=transform.kms_key))
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_01_16-1655655532510206480?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_08_33-1949775218156178510?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_17_43-7158254710436991981?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_27-15471683801547441211?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:680:
BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use
WriteToBigQuery instead.
kms_key=transform.kms_key))
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:73:
BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use
WriteToBigQuery instead.
kms_key=kms_key))
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_51_54-2548097743986743126?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_59_06-14932226357052494359?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_06_49-8949554077575390690?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_14_41-8460379551398592859?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_30-15639551169303289995?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1137:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
method_to_use = self._compute_method(p, p.options)
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:545:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_52_01-15275042995928089772?project=apache-beam-testing.
or p.options.view_as(GoogleCloudOptions).temp_location)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_02_01-13342647003031795811?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_09_50-1728655561156102290?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_41_29-15707264661757982472?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_10_50_58-16158069906186278177?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_00_12-4970334536023500649?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_08_18-8399946114541766616?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_17_59-419717168236828277?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-03_11_25_45-484746509541652528?project=apache-beam-testing.
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 41 tests in 3191.800s
FAILED (SKIP=4, errors=1)
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/test-suites/direct/py35/build.gradle'>
line: 48
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py35:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/test-suites/dataflow/py35/build.gradle'>
line: 48
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py35:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/test-suites/dataflow/py36/build.gradle'>
line: 48
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python3_Verify/ws/src/sdks/python/test-suites/dataflow/py37/build.gradle'>
line: 78
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 54m 46s
77 actionable tasks: 60 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/2lvplrgdtsjay
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
