See
<https://builds.apache.org/job/beam_PostCommit_Python2/2415/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9164] Re-enable UnboundedSourceWrapper#testWatermarkEmission test
[github] Merge pull request #11673 from [BEAM-9967] Adding support for BQ labels
[github] [BEAM-9622] Add Python SqlTransform test that joins tagged PCollections
------------------------------------------
[...truncated 12.90 MB...]
File "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 744, in _execute_query
job_labels=self.bigquery_job_labels)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 249, in wrapper
raise_with_traceback(exn, exn_traceback)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 236, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
line 415, in _start_query_job
labels=job_labels or {},
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 973, in __setattr__
object.__setattr__(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 1651, in __set__
value = t(**value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 976, in __setattr__
"to message %s" % (name, type(self).__name__))
AttributeError: May not assign arbitrary value owner to message LabelsValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:30:11.820Z:
JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service
Account.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:30:22.211Z:
JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/batchworker.py",
line 647, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 226, in execute
self._split_task)
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 698, in split
self.table_reference = self._execute_query(bq)
File
"/usr/local/lib/python2.7/site-packages/apache_beam/options/value_provider.py",
line 135, in _f
return fnc(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 744, in _execute_query
job_labels=self.bigquery_job_labels)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 249, in wrapper
raise_with_traceback(exn, exn_traceback)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 236, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
line 415, in _start_query_job
labels=job_labels or {},
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 973, in __setattr__
object.__setattr__(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 1651, in __set__
value = t(**value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 976, in __setattr__
"to message %s" % (name, type(self).__name__))
AttributeError: May not assign arbitrary value owner to message LabelsValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:30:46.971Z:
JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/batchworker.py",
line 647, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 226, in execute
self._split_task)
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 698, in split
self.table_reference = self._execute_query(bq)
File
"/usr/local/lib/python2.7/site-packages/apache_beam/options/value_provider.py",
line 135, in _f
return fnc(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 744, in _execute_query
job_labels=self.bigquery_job_labels)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 249, in wrapper
raise_with_traceback(exn, exn_traceback)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 236, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
line 415, in _start_query_job
labels=job_labels or {},
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 973, in __setattr__
object.__setattr__(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 1651, in __set__
value = t(**value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 976, in __setattr__
"to message %s" % (name, type(self).__name__))
AttributeError: May not assign arbitrary value owner to message LabelsValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:10.616Z:
JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/batchworker.py",
line 647, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 226, in execute
self._split_task)
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py",
line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 698, in split
self.table_reference = self._execute_query(bq)
File
"/usr/local/lib/python2.7/site-packages/apache_beam/options/value_provider.py",
line 135, in _f
return fnc(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 744, in _execute_query
job_labels=self.bigquery_job_labels)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 249, in wrapper
raise_with_traceback(exn, exn_traceback)
File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py",
line 236, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
line 415, in _start_query_job
labels=job_labels or {},
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 973, in __setattr__
object.__setattr__(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 1651, in __set__
value = t(**value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",
line 976, in __setattr__
"to message %s" % (name, type(self).__name__))
AttributeError: May not assign arbitrary value owner to message LabelsValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:10.909Z:
JOB_MESSAGE_BASIC: Finished operation
read/Read+read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:10.982Z:
JOB_MESSAGE_DEBUG: Executing failure step failure21
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:11.018Z:
JOB_MESSAGE_ERROR: Workflow failed. Causes:
S02:read/Read+read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
failed., Internal Issue (e4b6a0f9e3701d18): 63963027:24514
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:11.096Z:
JOB_MESSAGE_BASIC: Finished operation
assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:11.209Z:
JOB_MESSAGE_DETAILED: Cleaning up.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:11.280Z:
JOB_MESSAGE_DEBUG: Starting worker pool teardown.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:31:11.315Z:
JOB_MESSAGE_BASIC: Stopping worker pool...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:32:33.789Z:
JOB_MESSAGE_DETAILED: Autoscaling: Resized worker pool from 1 to 0.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:32:33.836Z:
JOB_MESSAGE_BASIC: Worker pool stopped.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T00:32:33.906Z:
JOB_MESSAGE_DEBUG: Tearing down pending resources...
apache_beam.runners.dataflow.dataflow_runner: INFO: Job
2020-05-12_17_24_04-17964336553168023617 is in state JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 58 tests in 4021.523s
FAILED (SKIP=7, errors=1)
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_05_51-6563613807928111996?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_25_19-9323330599522857346?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_32_58-4604475716432937206?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_40_36-8488782306687114997?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_48_09-13480099468016612711?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_06_00-2483592254372985846?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_20_42-3430406041217513639?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_29_32-4988335302646368408?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_38_05-9097021390401749607?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_45_41-15905669998912281561?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_52_50-3329561380358212076?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_05_59-12784098064315423472?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_18_18-2419079640019456097?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_25_34-2045205191521658539?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_33_24-16908263418850741193?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_41_14-728790960408423476?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_48_54-5704081668256415804?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_56_55-13232111961472452712?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_05_52-11028682513097053190?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_25_42-9692358668631331614?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_33_32-5894525427869991198?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_41_26-3249222648755049109?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_49_37-3402355421471473244?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_05_55-5395217480672630135?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_14_32-14067318974842369524?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_24_04-17964336553168023617?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_33_02-15356803296995225459?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_41_16-581485624139094956?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_49_19-16220744231626464877?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_56_54-11579037594944922780?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_05_52-12031924054074573304?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_14_34-6387054370934549474?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_23_06-698077014632476125?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_32_52-8810619824281152061?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_41_51-15581574515550329749?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_49_30-1184256058999801407?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_56_49-2812218497353514523?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_18_04_32-9678075321277588826?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_05_55-16891255141343703169?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_15_35-17136834610986319098?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_23_36-5084869389988730704?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_31_36-3170114667996907456?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_39_12-2808491533373021552?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_47_49-16457865423827450448?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_55_04-8887459957664299567?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_05_55-7294731269972970574?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_16_55-6340146698089045943?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_27_40-5794785342074501986?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_36_27-17200541417596810191?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_54_10-16048363235339332795?project=apache-beam-testing
> Task :sdks:python:test-suites:dataflow:py2:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/portable/py2/build.gradle'>
line: 131
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py2:postCommitPy2IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/direct/py2/build.gradle'>
line: 50
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py2:directRunnerIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/dataflow/py2/build.gradle'>
line: 116
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py2:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 9m 22s
123 actionable tasks: 100 executed, 21 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/wd4lgjw2yotly
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
