See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/3537/display/redirect?page=changes>
Changes:
[dannymccormick] Update dashboards to use gh data instead of jira data
[noreply] Merge pull request #21746: Exclude GCP Java packages from Dependabot
[noreply] Update
.test-infra/metrics/grafana/dashboards/source_data_freshness.json
[noreply] Better cross langauge support for dataframe reads. (#21762)
[noreply] Add template_location flag to Go Dataflow runner (#21774)
[noreply] [BEAM-14406] Drain test for SDF truncation in Go SDK (#17814)
[noreply] More Jira -> Issues doc updates (#21770)
[noreply] [BEAM-11104] Add code snippet for Go SDK Self-Checkpointing (#17956)
------------------------------------------
[...truncated 99.70 KB...]
supports_download: False>
http_response = Response(info={'vary': 'Origin, X-Origin, Referer',
'content-type': 'application/json; charset=UTF-8', 'date': 'Fri,
1...apis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2022-06-09_20_45_35-910284665965626872?alt=json')
request = <DataflowProjectsLocationsJobsUpdateRequest
job: <Job
requestedState: RequestedStateValueValuesEnum(JOB_STATE_CANCEL...Files: []>
jobId: '2022-06-09_20_45_35-910284665965626872'
location: 'us-central1'
projectId: 'apache-beam-testing'>
[1m def __ProcessHttpResponse(self, method_config, http_response,
request):[0m
[1m """Process the given http response."""[0m
[1m if http_response.status_code not in (http_client.OK,[0m
[1m http_client.CREATED,[0m
[1m http_client.NO_CONTENT):[0m
[1m> raise exceptions.HttpError.FromResponse([0m
[1m http_response, method_config=method_config,
request=request)[0m
[1m[31mE apitools.base.py.exceptions.HttpBadRequestError: HttpError
accessing
<https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2022-06-09_20_45_35-910284665965626872?alt=json>:
response: <{'vary': 'Origin, X-Origin, Referer', 'content-type':
'application/json; charset=UTF-8', 'date': 'Fri, 10 Jun 2022 03:55:41 GMT',
'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0',
'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff',
'transfer-encoding': 'chunked', 'status': '400', 'content-length': '314',
'-content-encoding': 'gzip'}>, content <{[0m
[1m[31mE "error": {[0m
[1m[31mE "code": 400,[0m
[1m[31mE "message": "(9bffe65afb6658e4): Workflow modification
failed. Causes: (38071fff39dc1f32): Operation cancel not allowed for job
2022-06-09_20_45_35-910284665965626872. Job is not yet ready for canceling.
Please retry in a few minutes.",[0m
[1m[31mE "status": "FAILED_PRECONDITION"[0m
[1m[31mE }[0m
[1m[31mE }[0m
[1m[31mE >[0m
[1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/apitools/base/py/base_api.py[0m:603:
HttpBadRequestError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:754
Executing command:
['<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/bin/python3.9',>
'-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r',
'/tmp/tmppxxpvzif/tmp_requirements.txt', '--exists-action', 'i', '--no-deps',
'--implementation', 'cp', '--abi', 'cp39', '--platform', 'manylinux2014_x86_64']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:325 Copying
Beam SDK
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/build/apache-beam.tar.gz";>
to staging location.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:476 Pipeline
has additional dependencies to be installed in SDK worker container, consider
using the SDK container image pre-building workflow to avoid repetitive
installations. Learn more on
https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
[32mINFO [0m root:environments.py:376 Default Python SDK image for
environment is apache/beam_python3.9_sdk:2.40.0.dev
[32mINFO [0m root:environments.py:295 Using provided Python SDK container
image: gcr.io/cloud-dataflow/v1beta3/python39-fnapi:beam-master-20220512
[32mINFO [0m root:environments.py:302 Python SDK container image set to
"gcr.io/cloud-dataflow/v1beta3/python39-fnapi:beam-master-20220512" for Docker
environment
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:136 Setting socket
default timeout to 60 seconds.
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:138 socket default
timeout is 60.0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/requirements.txt...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/requirements.txt
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/mock-2.0.0-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/mock-2.0.0-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/seaborn-0.11.2-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/seaborn-0.11.2-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/PyHamcrest-1.10.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/PyHamcrest-1.10.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/beautifulsoup4-4.11.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/beautifulsoup4-4.11.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/parameterized-0.7.5-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/parameterized-0.7.5-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/matplotlib-3.5.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/matplotlib-3.5.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/matplotlib-3.5.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/matplotlib-3.5.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/dataflow_python_sdk.tar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/dataflow_python_sdk.tar
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/dataflow-worker.jar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/dataflow-worker.jar
in 6 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/pipeline.pb...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034526-810997-zxz6xnth.1654832726.811192/pipeline.pb
in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:335
Discarding unparseable args: ['--sleep_secs=20']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:335
Discarding unparseable args: ['--sleep_secs=20']
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:897 Create job:
<Job
clientRequestId: '20220610034526812330-5768'
createTime: '2022-06-10T03:45:36.395042Z'
currentStateTime: '1970-01-01T00:00:00Z'
id:
'2022-06-09_20_45_35-910284665965626872'
location: 'us-central1'
name: 'beamapp-jenkins-0610034526-810997-zxz6xnth'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-06-10T03:45:36.395042Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:899 Created job
with id: [2022-06-09_20_45_35-910284665965626872]
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:900 Submitted job:
2022-06-09_20_45_35-910284665965626872
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:901 To access the
Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-06-09_20_45_35-910284665965626872?project=apache-beam-testing
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58
Console log:
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-06-09_20_45_35-910284665965626872?project=apache-beam-testing
[1m[31mERROR [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:102
Console URL:
https://console.cloud.google.com/dataflow/<regionId>/2022-06-09_20_45_35-910284665965626872?project=<projectId>
[31m[1m___________________ PTransformTest.test_flatten_pcollections
___________________[0m
[gw4] linux -- Python 3.9.10
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/bin/python3.9>
self = <apache_beam.transforms.ptransform_test.PTransformTest
testMethod=test_flatten_pcollections>
[1m @pytest.mark.it_validatesrunner[0m
[1m def test_flatten_pcollections(self):[0m
[1m with TestPipeline() as pipeline:[0m
[1m pcoll_1 = pipeline | 'Start 1' >> beam.Create([0, 1, 2, 3])[0m
[1m pcoll_2 = pipeline | 'Start 2' >> beam.Create([4, 5, 6, 7])[0m
[1m result = (pcoll_1, pcoll_2) | 'Flatten' >> beam.Flatten()[0m
[1m> assert_that(result, equal_to([0, 1, 2, 3, 4, 5, 6, 7]))[0m
[1m[31mapache_beam/transforms/ptransform_test.py[0m:686:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/pipeline.py[0m:596: in __exit__
[1m self.result = self.run()[0m
[1m[31mapache_beam/testing/test_pipeline.py[0m:112: in run
[1m result = super().run([0m
[1m[31mapache_beam/pipeline.py[0m:546: in run
[1m return Pipeline.from_runner_api([0m
[1m[31mapache_beam/pipeline.py[0m:573: in run
[1m return self.runner.run_pipeline(self, self._options)[0m
[1m[31mapache_beam/runners/dataflow/test_dataflow_runner.py[0m:73: in
run_pipeline
[1m self.result.cancel()[0m
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:1691: in cancel
[1m if not self._runner.dataflow_client.modify_job_state([0m
[1m[31mapache_beam/utils/retry.py[0m:275: in wrapper
[1m return fun(*args, **kwargs)[0m
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:938: in
modify_job_state
[1m self._client.projects_locations_jobs.Update(request)[0m
[1m[31mapache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py[0m:857:
in Update
[1m return self._RunMethod([0m
[1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/apitools/base/py/base_api.py[0m:731:
in _RunMethod
[1m return self.ProcessHttpResponse(method_config, http_response,
request)[0m
[1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/apitools/base/py/base_api.py[0m:737:
in ProcessHttpResponse
[1m self.__ProcessHttpResponse(method_config, http_response, request))[0m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
<apache_beam.runners.dataflow.internal.clients.dataflow.dataflow_v1b3_client.DataflowV1b3.ProjectsLocationsJobsService
object at 0x7f7408736fa0>
method_config = <ApiMethodInfo
relative_path: 'v1b3/projects/{projectId}/locations/{location}/jobs/{jobId}'
method_id: 'dataflow.pro...DataflowProjectsLocationsJobsUpdateRequest'
response_type_name: 'Job'
request_field: 'job'
supports_download: False>
http_response = Response(info={'vary': 'Origin, X-Origin, Referer',
'content-type': 'application/json; charset=UTF-8', 'date': 'Fri,
1...is.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2022-06-09_20_45_36-11245425781196672726?alt=json')
request = <DataflowProjectsLocationsJobsUpdateRequest
job: <Job
requestedState: RequestedStateValueValuesEnum(JOB_STATE_CANCEL...les: []>
jobId: '2022-06-09_20_45_36-11245425781196672726'
location: 'us-central1'
projectId: 'apache-beam-testing'>
[1m def __ProcessHttpResponse(self, method_config, http_response,
request):[0m
[1m """Process the given http response."""[0m
[1m if http_response.status_code not in (http_client.OK,[0m
[1m http_client.CREATED,[0m
[1m http_client.NO_CONTENT):[0m
[1m> raise exceptions.HttpError.FromResponse([0m
[1m http_response, method_config=method_config,
request=request)[0m
[1m[31mE apitools.base.py.exceptions.HttpBadRequestError: HttpError
accessing
<https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2022-06-09_20_45_36-11245425781196672726?alt=json>:
response: <{'vary': 'Origin, X-Origin, Referer', 'content-type':
'application/json; charset=UTF-8', 'date': 'Fri, 10 Jun 2022 03:55:42 GMT',
'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0',
'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff',
'transfer-encoding': 'chunked', 'status': '400', 'content-length': '316',
'-content-encoding': 'gzip'}>, content <{[0m
[1m[31mE "error": {[0m
[1m[31mE "code": 400,[0m
[1m[31mE "message": "(7a333ff2a333b817): Workflow modification
failed. Causes: (62f7b1f095ecdaad): Operation cancel not allowed for job
2022-06-09_20_45_36-11245425781196672726. Job is not yet ready for canceling.
Please retry in a few minutes.",[0m
[1m[31mE "status": "FAILED_PRECONDITION"[0m
[1m[31mE }[0m
[1m[31mE }[0m
[1m[31mE >[0m
[1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/apitools/base/py/base_api.py[0m:603:
HttpBadRequestError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:754
Executing command:
['<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/bin/python3.9',>
'-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r',
'/tmp/tmpnveyac9i/tmp_requirements.txt', '--exists-action', 'i', '--no-deps',
'--implementation', 'cp', '--abi', 'cp39', '--platform', 'manylinux2014_x86_64']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:325 Copying
Beam SDK
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/build/apache-beam.tar.gz";>
to staging location.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:476 Pipeline
has additional dependencies to be installed in SDK worker container, consider
using the SDK container image pre-building workflow to avoid repetitive
installations. Learn more on
https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
[32mINFO [0m root:environments.py:376 Default Python SDK image for
environment is apache/beam_python3.9_sdk:2.40.0.dev
[32mINFO [0m root:environments.py:295 Using provided Python SDK container
image: gcr.io/cloud-dataflow/v1beta3/python39-fnapi:beam-master-20220512
[32mINFO [0m root:environments.py:302 Python SDK container image set to
"gcr.io/cloud-dataflow/v1beta3/python39-fnapi:beam-master-20220512" for Docker
environment
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:136 Setting socket
default timeout to 60 seconds.
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:138 socket default
timeout is 60.0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/requirements.txt...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/requirements.txt
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/mock-2.0.0-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/mock-2.0.0-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/seaborn-0.11.2-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/seaborn-0.11.2-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/PyHamcrest-1.10.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/PyHamcrest-1.10.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/beautifulsoup4-4.11.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/beautifulsoup4-4.11.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/parameterized-0.7.5-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/parameterized-0.7.5-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/matplotlib-3.5.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/matplotlib-3.5.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/matplotlib-3.5.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/matplotlib-3.5.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/dataflow_python_sdk.tar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/dataflow_python_sdk.tar
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/dataflow-worker.jar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/dataflow-worker.jar
in 5 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:718 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/pipeline.pb...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:734 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0610034527-988009-ge34as1h.1654832727.988202/pipeline.pb
in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:335
Discarding unparseable args: ['--sleep_secs=20']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:335
Discarding unparseable args: ['--sleep_secs=20']
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:897 Create job:
<Job
clientRequestId: '20220610034527989327-9144'
createTime: '2022-06-10T03:45:37.427888Z'
currentStateTime: '1970-01-01T00:00:00Z'
id:
'2022-06-09_20_45_36-11245425781196672726'
location: 'us-central1'
name: 'beamapp-jenkins-0610034527-988009-ge34as1h'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-06-10T03:45:37.427888Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:899 Created job
with id: [2022-06-09_20_45_36-11245425781196672726]
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:900 Submitted job:
2022-06-09_20_45_36-11245425781196672726
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:901 To access the
Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-06-09_20_45_36-11245425781196672726?project=apache-beam-testing
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58
Console log:
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-06-09_20_45_36-11245425781196672726?project=apache-beam-testing
[31m[1mERROR [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:102
Console URL:
https://console.cloud.google.com/dataflow/<regionId>/2022-06-09_20_45_36-11245425781196672726?project=<projectId>
[33m=============================== warnings summary
===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py39-xdist.xml>
-
[31m[1m======== 3 failed, 29 passed, 2 skipped, 8 warnings in 2347.58 seconds
=========[0m
> Task :sdks:python:test-suites:dataflow:py39:validatesRunnerStreamingTests
> FAILED
FAILURE: Build failed with an exception.
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 264
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py39:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 15m 41s
93 actionable tasks: 58 executed, 33 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/23qdqwqyqaeuk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
