See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/2415/display/redirect>
Changes: ------------------------------------------ [...truncated 92.41 KB...] [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/PyHamcrest-1.10.1-py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/PyHamcrest-1.10.1-py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/parameterized-0.7.5-py2.py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/parameterized-0.7.5-py2.py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/pipeline.pb... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005010659-578944-mwbf9kot.1696468019.579116/pipeline.pb in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:843 Create job: <Job clientRequestId: '20231005010659579868-5807' createTime: '2023-10-05T01:07:01.955329Z' currentStateTime: '1970-01-01T00:00:00Z' id: '2023-10-04_18_07_01-17359288325379354658' location: 'us-central1' name: 'beamapp-jenkins-1005010659-578944-mwbf9kot' projectId: 'apache-beam-testing' stageStates: [] startTime: '2023-10-05T01:07:01.955329Z' steps: [] tempFiles: [] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:845 Created job with id: [2023-10-04_18_07_01-17359288325379354658] [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:846 Submitted job: 2023-10-04_18_07_01-17359288325379354658 [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:847 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-04_18_07_01-17359288325379354658?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58 Console log: [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59 https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-04_18_07_01-17359288325379354658?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-10-04_18_07_01-17359288325379354658 is in state JOB_STATE_RUNNING [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:07:05.054Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:07:07.396Z: JOB_MESSAGE_BASIC: Executing operation ReadFromText/Read/Impulse+ReadFromText/Read/EmitSource+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/PairWithRestriction+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/SplitWithSizing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:07:07.423Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/DoOnce/Impulse+WriteToText/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:3759>)+WriteToText/Write/WriteImpl/DoOnce/Map(decode)+WriteToText/Write/WriteImpl/InitializeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:07:07.454Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-c... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:07:24.554Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:00.098Z: JOB_MESSAGE_BASIC: All workers have finished the startup processes and began to receive work requests. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.622Z: JOB_MESSAGE_BASIC: Finished operation ReadFromText/Read/Impulse+ReadFromText/Read/EmitSource+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/PairWithRestriction+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/SplitWithSizing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.623Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/DoOnce/Impulse+WriteToText/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:3759>)+WriteToText/Write/WriteImpl/DoOnce/Map(decode)+WriteToText/Write/WriteImpl/InitializeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.741Z: JOB_MESSAGE_BASIC: Executing operation Count1/CombinePerKey(sum)/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.770Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/WriteBundles/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.791Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.812Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.820Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/WriteBundles/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.841Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:01.865Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:02.152Z: JOB_MESSAGE_BASIC: Finished operation Count1/CombinePerKey(sum)/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:02.247Z: JOB_MESSAGE_BASIC: Executing operation ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/ProcessElementAndRestrictionWithSizing+Count1/ParWithOne+Count1/CombinePerKey(sum)/GroupByKey+Count1/CombinePerKey(sum)/Combine/Partial+Count1/CombinePerKey(sum)/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:03.346Z: JOB_MESSAGE_BASIC: Finished operation ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/ProcessElementAndRestrictionWithSizing+Count1/ParWithOne+Count1/CombinePerKey(sum)/GroupByKey+Count1/CombinePerKey(sum)/Combine/Partial+Count1/CombinePerKey(sum)/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:03.390Z: JOB_MESSAGE_BASIC: Executing operation Count1/CombinePerKey(sum)/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:03.436Z: JOB_MESSAGE_BASIC: Finished operation Count1/CombinePerKey(sum)/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:03.509Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:03.640Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:03.735Z: JOB_MESSAGE_BASIC: Executing operation Count1/CombinePerKey(sum)/GroupByKey/Read+Count1/CombinePerKey(sum)/Combine+Count1/CombinePerKey(sum)/Combine/Extract+WriteToText/Write/WriteImpl/WindowInto(WindowIntoFn)+WriteToText/Write/WriteImpl/WriteBundles+WriteToText/Write/WriteImpl/Pair+WriteToText/Write/WriteImpl/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:04.525Z: JOB_MESSAGE_BASIC: Finished operation Count1/CombinePerKey(sum)/GroupByKey/Read+Count1/CombinePerKey(sum)/Combine+Count1/CombinePerKey(sum)/Combine/Extract+WriteToText/Write/WriteImpl/WindowInto(WindowIntoFn)+WriteToText/Write/WriteImpl/WriteBundles+WriteToText/Write/WriteImpl/Pair+WriteToText/Write/WriteImpl/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:04.570Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:04.619Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:04.665Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/GroupByKey/Read+WriteToText/Write/WriteImpl/Extract [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:06.288Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/GroupByKey/Read+WriteToText/Write/WriteImpl/Extract [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:06.408Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:06.431Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:06.461Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:06.490Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:06.578Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/PreFinalize [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:08.647Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/PreFinalize [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:08.792Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input2 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:08.842Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input2 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:08.967Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:10.529Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:10:10.845Z: JOB_MESSAGE_BASIC: Stopping worker pool... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:12:16.188Z: JOB_MESSAGE_BASIC: Worker pool stopped. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-10-04_18_07_01-17359288325379354658 is in state JOB_STATE_DONE [32mINFO [0m root:custom_ptransform.py:67 Running second pipeline [32mINFO [0m apache_beam.runners.portability.stager:stager.py:761 Executing command: ['<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/build/gradleenv/2050596099/bin/python3.11',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', '/tmp/tmpq2636hf1/tmp_requirements.txt', '--exists-action', 'i', '--no-deps', '--implementation', 'cp', '--abi', 'cp311', '--platform', 'manylinux2014_x86_64'] [32mINFO [0m apache_beam.runners.portability.stager:stager.py:322 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/sdks/python/build/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl";> to staging location. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:395 Pipeline has additional dependencies to be installed in SDK worker container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild [32mINFO [0m root:environments.py:313 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:beam-master-20230927 [32mINFO [0m root:environments.py:320 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:beam-master-20230927" for Docker environment [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function pack_combiners at 0x7f46c3724ea0> ==================== [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function sort_stages at 0x7f46c3725760> ==================== [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/requirements.txt... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/requirements.txt in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/mock-2.0.0-py2.py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/mock-2.0.0-py2.py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/PyHamcrest-1.10.1-py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/PyHamcrest-1.10.1-py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/parameterized-0.7.5-py2.py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/parameterized-0.7.5-py2.py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl in 1 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/pipeline.pb... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011228-614726-0k4kdurm.1696468348.614891/pipeline.pb in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:843 Create job: <Job clientRequestId: '20231005011228615665-6630' createTime: '2023-10-05T01:12:31.084967Z' currentStateTime: '1970-01-01T00:00:00Z' id: '2023-10-04_18_12_30-4303669878673965179' location: 'us-central1' name: 'beamapp-jenkins-1005011228-614726-0k4kdurm' projectId: 'apache-beam-testing' stageStates: [] startTime: '2023-10-05T01:12:31.084967Z' steps: [] tempFiles: [] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:845 Created job with id: [2023-10-04_18_12_30-4303669878673965179] [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:846 Submitted job: 2023-10-04_18_12_30-4303669878673965179 [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:847 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-04_18_12_30-4303669878673965179?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58 Console log: [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59 https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-04_18_12_30-4303669878673965179?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-10-04_18_12_30-4303669878673965179 is in state JOB_STATE_RUNNING [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:12:34.428Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:12:36.515Z: JOB_MESSAGE_BASIC: Executing operation ReadFromText/Read/Impulse+ReadFromText/Read/EmitSource+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/PairWithRestriction+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/SplitWithSizing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:12:36.540Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/DoOnce/Impulse+WriteToText/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:3759>)+WriteToText/Write/WriteImpl/DoOnce/Map(decode)+WriteToText/Write/WriteImpl/InitializeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:12:36.581Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-c... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:13:04.464Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:41.199Z: JOB_MESSAGE_BASIC: All workers have finished the startup processes and began to receive work requests. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.511Z: JOB_MESSAGE_BASIC: Finished operation ReadFromText/Read/Impulse+ReadFromText/Read/EmitSource+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/PairWithRestriction+ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/SplitWithSizing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.649Z: JOB_MESSAGE_BASIC: Executing operation Count2/CombinePerKey(sum)/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.664Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/DoOnce/Impulse+WriteToText/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:3759>)+WriteToText/Write/WriteImpl/DoOnce/Map(decode)+WriteToText/Write/WriteImpl/InitializeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.789Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/WriteBundles/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.812Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.834Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.836Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/WriteBundles/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.860Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:42.883Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input0 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:43.277Z: JOB_MESSAGE_BASIC: Finished operation Count2/CombinePerKey(sum)/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:43.373Z: JOB_MESSAGE_BASIC: Executing operation ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/ProcessElementAndRestrictionWithSizing+Count2/PairWithOne+Count2/CombinePerKey(sum)/GroupByKey+Count2/CombinePerKey(sum)/Combine/Partial+Count2/CombinePerKey(sum)/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:44.481Z: JOB_MESSAGE_BASIC: Finished operation ref_AppliedPTransform_ReadFromText-Read-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_7/ProcessElementAndRestrictionWithSizing+Count2/PairWithOne+Count2/CombinePerKey(sum)/GroupByKey+Count2/CombinePerKey(sum)/Combine/Partial+Count2/CombinePerKey(sum)/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:44.591Z: JOB_MESSAGE_BASIC: Executing operation Count2/CombinePerKey(sum)/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:44.664Z: JOB_MESSAGE_BASIC: Finished operation Count2/CombinePerKey(sum)/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:44.712Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:44.845Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:44.954Z: JOB_MESSAGE_BASIC: Executing operation Count2/CombinePerKey(sum)/GroupByKey/Read+Count2/CombinePerKey(sum)/Combine+Count2/CombinePerKey(sum)/Combine/Extract+WriteToText/Write/WriteImpl/WindowInto(WindowIntoFn)+WriteToText/Write/WriteImpl/WriteBundles+WriteToText/Write/WriteImpl/Pair+WriteToText/Write/WriteImpl/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:45.869Z: JOB_MESSAGE_BASIC: Finished operation Count2/CombinePerKey(sum)/GroupByKey/Read+Count2/CombinePerKey(sum)/Combine+Count2/CombinePerKey(sum)/Combine/Extract+WriteToText/Write/WriteImpl/WindowInto(WindowIntoFn)+WriteToText/Write/WriteImpl/WriteBundles+WriteToText/Write/WriteImpl/Pair+WriteToText/Write/WriteImpl/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:45.918Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:45.965Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:46.016Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/GroupByKey/Read+WriteToText/Write/WriteImpl/Extract [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:47.599Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/GroupByKey/Read+WriteToText/Write/WriteImpl/Extract [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:47.704Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:47.722Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:47.748Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:47.785Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/PreFinalize/View-python_side_input1 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:47.893Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/PreFinalize [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:50.550Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/PreFinalize [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:50.647Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input2 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:50.695Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite/View-python_side_input2 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:50.799Z: JOB_MESSAGE_BASIC: Executing operation WriteToText/Write/WriteImpl/FinalizeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:52.301Z: JOB_MESSAGE_BASIC: Finished operation WriteToText/Write/WriteImpl/FinalizeWrite [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:15:52.609Z: JOB_MESSAGE_BASIC: Stopping worker pool... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-10-05T01:18:01.891Z: JOB_MESSAGE_BASIC: Worker pool stopped. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-10-04_18_12_30-4303669878673965179 is in state JOB_STATE_DONE [32mINFO [0m root:custom_ptransform.py:94 Running third pipeline [32mINFO [0m apache_beam.runners.portability.stager:stager.py:761 Executing command: ['<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/build/gradleenv/2050596099/bin/python3.11',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', '/tmp/tmp9jr39a02/tmp_requirements.txt', '--exists-action', 'i', '--no-deps', '--implementation', 'cp', '--abi', 'cp311', '--platform', 'manylinux2014_x86_64'] [32mINFO [0m apache_beam.runners.portability.stager:stager.py:322 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/sdks/python/build/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl";> to staging location. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:395 Pipeline has additional dependencies to be installed in SDK worker container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild [32mINFO [0m root:environments.py:313 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:beam-master-20230927 [32mINFO [0m root:environments.py:320 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:beam-master-20230927" for Docker environment [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function pack_combiners at 0x7f46c3724ea0> ==================== [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function sort_stages at 0x7f46c3725760> ==================== [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/requirements.txt... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/requirements.txt in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/mock-2.0.0-py2.py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/mock-2.0.0-py2.py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/PyHamcrest-1.10.1-py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/PyHamcrest-1.10.1-py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/parameterized-0.7.5-py2.py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/parameterized-0.7.5-py2.py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/pipeline.pb... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1005011812-808924-yffu39d3.1696468692.809098/pipeline.pb in 0 seconds. [33m=============================== warnings summary ===============================[0m ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py>:28: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses from imp import load_source ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 ../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import util apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead. table_ref = client.dataset(dataset_id).table(table_id) apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead. dataset_ref = client.dataset(unique_dataset_name, project=project) apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:47: FutureWarning: The default value of numeric_only in DataFrame.mean is deprecated. In a future version, it will default to False. In addition, specifying 'numeric_only=None' is deprecated. Select only valid columns or specify the value of numeric_only to silence this warning. return airline_df[at_top_airports].mean() -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/sdks/python/pytest_postCommitIT-df-py311-xdist.xml> - [36m[1m=========================== short test summary info ============================[0m [31mFAILED[0m apache_beam/examples/cookbook/custom_ptransform_it_test.py::[1mCustomPTransformIT::test_custom_ptransform_output_files_on_small_input[0m - apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 05 Oct 2023 01:18:15 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '492', '-content-encoding': 'gzip'}>, content <{ "error": { "code": 400, "message": "(a98ce6fd4bc70fa3): The workflow could not be created. Causes: (b49ec3e0a613285a): Dataflow quota error for jobs-per-project quota. Project apache-beam-testing is running 300 jobs. Please check the quota usage via GCP Console. If it exceeds the limit, please wait for a workflow to finish or contact Google Cloud Support to request an increase in quota. If it does not, contact Google Cloud Support.", "status": "FAILED_PRECONDITION" } } > [31m====== [31m[1m1 failed[0m, [32m14 passed[0m, [33m17 skipped[0m, [33m22 warnings[0m[31m in 1265.97s (0:21:05)[0m[31m ======[0m > Task :sdks:python:test-suites:dataflow:py311:examples FAILED FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 220 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py311:examples'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Get more help at https://help.gradle.org. Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. BUILD FAILED in 26m 12s 11 actionable tasks: 10 executed, 1 from cache Publishing build scan... https://ge.apache.org/s/fh42ka2gidroi Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
