See
<https://builds.apache.org/job/beam_PostCommit_Python37/2343/display/redirect?page=changes>
Changes:
[kawaigin] Use csv reader instead of split to read csv data.
------------------------------------------
[...truncated 12.24 MB...]
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py",
line 647, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 698, in split
self.table_reference = self._execute_query(bq)
File
"/usr/local/lib/python3.7/site-packages/apache_beam/options/value_provider.py",
line 135, in _f
return fnc(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 744, in _execute_query
job_labels=self.bigquery_job_labels)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 249, in wrapper
raise_with_traceback(exn, exn_traceback)
File "/usr/local/lib/python3.7/site-packages/future/utils/__init__.py", line
446, in raise_with_traceback
raise exc.with_traceback(traceback)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 236, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
line 415, in _start_query_job
labels=job_labels or {},
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 973, in __setattr__
object.__setattr__(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 1651, in __set__
value = t(**value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 976, in __setattr__
"to message %s" % (name, type(self).__name__))
AttributeError: May not assign arbitrary value owner to message LabelsValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:24.551Z:
JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 246, in wrapper
sleep_interval = next(retry_intervals)
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py",
line 647, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 698, in split
self.table_reference = self._execute_query(bq)
File
"/usr/local/lib/python3.7/site-packages/apache_beam/options/value_provider.py",
line 135, in _f
return fnc(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 744, in _execute_query
job_labels=self.bigquery_job_labels)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 249, in wrapper
raise_with_traceback(exn, exn_traceback)
File "/usr/local/lib/python3.7/site-packages/future/utils/__init__.py", line
446, in raise_with_traceback
raise exc.with_traceback(traceback)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 236, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
line 415, in _start_query_job
labels=job_labels or {},
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 973, in __setattr__
object.__setattr__(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 1651, in __set__
value = t(**value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 976, in __setattr__
"to message %s" % (name, type(self).__name__))
AttributeError: May not assign arbitrary value owner to message LabelsValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.328Z:
JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 246, in wrapper
sleep_interval = next(retry_intervals)
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py",
line 647, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 698, in split
self.table_reference = self._execute_query(bq)
File
"/usr/local/lib/python3.7/site-packages/apache_beam/options/value_provider.py",
line 135, in _f
return fnc(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py",
line 744, in _execute_query
job_labels=self.bigquery_job_labels)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 249, in wrapper
raise_with_traceback(exn, exn_traceback)
File "/usr/local/lib/python3.7/site-packages/future/utils/__init__.py", line
446, in raise_with_traceback
raise exc.with_traceback(traceback)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py",
line 236, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
line 415, in _start_query_job
labels=job_labels or {},
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 973, in __setattr__
object.__setattr__(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 1651, in __set__
value = t(**value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 791, in __init__
setattr(self, name, value)
File
"/usr/local/lib/python3.7/site-packages/apitools/base/protorpclite/messages.py",
line 976, in __setattr__
"to message %s" % (name, type(self).__name__))
AttributeError: May not assign arbitrary value owner to message LabelsValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.590Z:
JOB_MESSAGE_BASIC: Finished operation
read/Read+read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.664Z:
JOB_MESSAGE_DEBUG: Executing failure step failure21
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.701Z:
JOB_MESSAGE_ERROR: Workflow failed. Causes:
S02:read/Read+read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
failed., Internal Issue (ce656e080aa9f2dc): 63963027:24514
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.767Z:
JOB_MESSAGE_BASIC: Finished operation
assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.887Z:
JOB_MESSAGE_DETAILED: Cleaning up.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.956Z:
JOB_MESSAGE_DEBUG: Starting worker pool teardown.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:38:52.989Z:
JOB_MESSAGE_BASIC: Stopping worker pool...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:40:51.411Z:
JOB_MESSAGE_DETAILED: Autoscaling: Resized worker pool from 1 to 0.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:40:51.458Z:
JOB_MESSAGE_BASIC: Worker pool stopped.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-05-13T18:40:51.497Z:
JOB_MESSAGE_DEBUG: Tearing down pending resources...
apache_beam.runners.dataflow.dataflow_runner: INFO: Job
2020-05-13_11_31_42-14582734305423653448 is in state JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_13-2202198591665578616?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_27_48-8328082657524195713?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_36_03-15853844704776128069?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_44_25-18402841262435804970?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_52_42-12435772465692621079?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_01_02-7092784057036099408?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_08-6700446756202105679?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_33_22-10340633463802869134?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_42_26-8914089967877262398?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_52_32-9147681579681242175?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_01_04-8416814175021414556?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_12-648574654433819396?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_26_09-14348707360107767279?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_34_00-16485553242629574841?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_42_58-9802370954133023112?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_51_27-7883850467043048034?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_59_16-6612321802152038222?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_08-7592448994346772955?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_31_21-3722654782695117121?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_39_40-7542997155562477450?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_47_39-15910448334986743291?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_56_21-18439983453335569835?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_04_48-6501035487823028472?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_10-9506970642045304051?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_21_32-78829502552186743?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_30_07-8608678644047439497?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_39_42-13384775377131865739?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_48_55-10143701425117007979?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_57_30-793233787955467994?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_05_51-1306136695145959525?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_09-14822423491542883713?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_21_34-15672226057389035039?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_30_15-5849802252186197533?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_38_41-9075811947383752690?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_48_39-18312746549588433020?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_56_10-11274964836919126944?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_04_21-11617406996852403966?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_13_25-10909525242741472436?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_09-6079795805919970017?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_22_28-15073691306040924564?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_31_42-14582734305423653448?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_41_20-177029134947407120?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_48_56-3257522942106956611?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_57_42-12215744264315145170?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_06_21-5598325572781155038?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_13_09-15753902920476428043?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_22_59-10001602262713519932?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_33_51-7106347167094961269?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_43_11-2437124533143777974?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_12_00_47-14565964867638867274?project=apache-beam-testing
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df-py37.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 58 tests in 4095.837s
FAILED (SKIP=7, errors=1)
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/py37/build.gradle'>
line: 59
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/py37/build.gradle'>
line: 51
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/py37/build.gradle'>
line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 10m 13s
87 actionable tasks: 64 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/fubo5zbj7yscm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
