See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/5521/display/redirect>
Changes: ------------------------------------------ [...truncated 58.41 MB...] [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/database.py[0m:235: in spanner_api [1m client_options=client_options,[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/gapic/spanner_client.py[0m:194: in __init__ [1m address=api_endpoint, channel=channel, credentials=credentials[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/gapic/transports/spanner_grpc_transport.py[0m:77: in __init__ [1m "grpc.keepalive_time_ms": _GRPC_KEEPALIVE_MS,[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/gapic/transports/spanner_grpc_transport.py[0m:114: in create_channel [1m address, credentials=credentials, scopes=cls._OAUTH_SCOPES, **kwargs[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/api_core/grpc_helpers.py[0m:297: in create_channel [1m return grpc.secure_channel(target, composite_credentials, **kwargs)[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/grpc/__init__.py[0m:2005: in secure_channel [1m credentials._credentials, compression)[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/grpc/_channel.py[0m:1480: in __init__ [1m credentials)[0m [1m[31msrc/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi[0m:454: in grpc._cython.cygrpc.Channel.__cinit__ [1m ???[0m [1m[31msrc/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi[0m:76: in grpc._cython.cygrpc._ChannelArgs.__cinit__ [1m ???[0m _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [1m> ???[0m [1m[31mE TypeError: Expected int, bytes, or behavior, got <class 'grpc_gcp_pb2.ApiConfig'>[0m [1m[31msrc/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi[0m:60: TypeError [31m[1m________________ SpannerWriteIntegrationTest.test_write_batches ________________[0m [gw1] linux -- Python 3.7.12 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/bin/python3.7> self = <apache_beam.io.gcp.experimental.spannerio_write_it_test.SpannerWriteIntegrationTest testMethod=test_write_batches> [1m @pytest.mark.spannerio_it[0m [1m def test_write_batches(self):[0m [1m _prefex = 'test_write_batches'[0m [1m mutations = [[0m [1m WriteMutation.insert([0m [1m 'Users', ('UserId', 'Key'), [(_prefex + '1', _prefex + 'inset-1')]),[0m [1m WriteMutation.insert([0m [1m 'Users', ('UserId', 'Key'), [(_prefex + '2', _prefex + 'inset-2')]),[0m [1m WriteMutation.insert([0m [1m 'Users', ('UserId', 'Key'), [(_prefex + '3', _prefex + 'inset-3')]),[0m [1m WriteMutation.insert([0m [1m 'Users', ('UserId', 'Key'), [(_prefex + '4', _prefex + 'inset-4')])[0m [1m ][0m [1m [0m [1m p = beam.Pipeline(argv=self.args)[0m [1m _ = ([0m [1m p | beam.Create(mutations) | WriteToSpanner([0m [1m project_id=self.project,[0m [1m instance_id=self.instance,[0m [1m database_id=self.TEST_DATABASE,[0m [1m max_batch_size_bytes=250))[0m [1m [0m [1m res = p.run()[0m [1m res.wait_until_finish()[0m [1m> self.assertEqual(self._count_data(_prefex), len(mutations))[0m [1m[31mapache_beam/io/gcp/experimental/spannerio_write_it_test.py[0m:139: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [1m[31mapache_beam/io/gcp/experimental/spannerio_write_it_test.py[0m:88: in _count_data [1m with database.snapshot() as snapshot:[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/database.py[0m:649: in __enter__ [1m session = self._session = self._database._pool.get()[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/pool.py[0m:273: in get [1m session.create()[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/session.py[0m:113: in create [1m api = self._database.spanner_api[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/database.py[0m:235: in spanner_api [1m client_options=client_options,[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/gapic/spanner_client.py[0m:194: in __init__ [1m address=api_endpoint, channel=channel, credentials=credentials[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/gapic/transports/spanner_grpc_transport.py[0m:77: in __init__ [1m "grpc.keepalive_time_ms": _GRPC_KEEPALIVE_MS,[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/cloud/spanner_v1/gapic/transports/spanner_grpc_transport.py[0m:114: in create_channel [1m address, credentials=credentials, scopes=cls._OAUTH_SCOPES, **kwargs[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/api_core/grpc_helpers.py[0m:297: in create_channel [1m return grpc.secure_channel(target, composite_credentials, **kwargs)[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/grpc/__init__.py[0m:2005: in secure_channel [1m credentials._credentials, compression)[0m [1m[31m../../build/gradleenv/-1734967052/lib/python3.7/site-packages/grpc/_channel.py[0m:1480: in __init__ [1m credentials)[0m [1m[31msrc/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi[0m:454: in grpc._cython.cygrpc.Channel.__cinit__ [1m ???[0m [1m[31msrc/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi[0m:76: in grpc._cython.cygrpc._ChannelArgs.__cinit__ [1m ???[0m _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [1m> ???[0m [1m[31mE TypeError: Expected int, bytes, or behavior, got <class 'grpc_gcp_pb2.ApiConfig'>[0m [1m[31msrc/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi[0m:60: TypeError ------------------------------ Captured log call ------------------------------- [32mINFO [0m apache_beam.runners.portability.stager:stager.py:754 Executing command: ['<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/bin/python3.7',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', '/tmp/tmpeb6rwtdm/tmp_requirements.txt', '--exists-action', 'i', '--no-deps', '--implementation', 'cp', '--abi', 'cp37m', '--platform', 'manylinux2014_x86_64'] [32mINFO [0m apache_beam.runners.portability.stager:stager.py:325 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz";> to staging location. [32mINFO [0m root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev [32mINFO [0m root:environments.py:296 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220617 [32mINFO [0m root:environments.py:304 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220617" for Docker environment [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function pack_combiners at 0x7f4ad01bd200> ==================== [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sort_stages at 0x7f4ad01bd9e0> ==================== [33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:573 Typical end users should not use this worker jar feature. It can only be used when FnAPI is enabled. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/requirements.txt... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/requirements.txt in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/pbr-5.9.0.tar.gz... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/pbr-5.9.0.tar.gz in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/mock-2.0.0.tar.gz... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/mock-2.0.0.tar.gz in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/six-1.16.0.tar.gz... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/six-1.16.0.tar.gz in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/PyHamcrest-1.10.1.tar.gz... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/PyHamcrest-1.10.1.tar.gz in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/parameterized-0.7.5.tar.gz... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/parameterized-0.7.5.tar.gz in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/mock-2.0.0-py2.py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/mock-2.0.0-py2.py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/seaborn-0.11.2-py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/seaborn-0.11.2-py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/PyHamcrest-1.10.1-py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/PyHamcrest-1.10.1-py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/beautifulsoup4-4.11.1-py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/beautifulsoup4-4.11.1-py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/parameterized-0.7.5-py2.py3-none-any.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/parameterized-0.7.5-py2.py3-none-any.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/matplotlib-3.5.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/matplotlib-3.5.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/dataflow_python_sdk.tar... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/dataflow_python_sdk.tar in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/matplotlib-3.5.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/matplotlib-3.5.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/dataflow-worker.jar... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/dataflow-worker.jar in 5 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:732 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/pipeline.pb... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:751 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0727093952-768680-m23eerbm.1658914792.768886/pipeline.pb in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:911 Create job: <Job clientRequestId: '20220727093952769887-3550' createTime: '2022-07-27T09:40:03.721301Z' currentStateTime: '1970-01-01T00:00:00Z' id: '2022-07-27_02_40_01-11545245067662907333' location: 'us-central1' name: 'beamapp-jenkins-0727093952-768680-m23eerbm' projectId: 'apache-beam-testing' stageStates: [] startTime: '2022-07-27T09:40:03.721301Z' steps: [] tempFiles: [] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:913 Created job with id: [2022-07-27_02_40_01-11545245067662907333] [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:914 Submitted job: 2022-07-27_02_40_01-11545245067662907333 [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:920 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-07-27_02_40_01-11545245067662907333?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58 Console log: [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59 https://console.cloud.google.com/dataflow/jobs/us-central1/2022-07-27_02_40_01-11545245067662907333?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:197 Job 2022-07-27_02_40_01-11545245067662907333 is in state JOB_STATE_RUNNING [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:04.150Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2022-07-27_02_40_01-11545245067662907333. The number of workers will be between 1 and 1000. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:04.263Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2022-07-27_02_40_01-11545245067662907333. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:05.957Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.658Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.693Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.727Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.767Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.807Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.831Z: JOB_MESSAGE_DETAILED: Unzipping flatten s5 for input s3.unbatchable [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.861Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of WriteToSpanner/Writing to spanner, through flatten WriteToSpanner/make batches/Merging batchable and unbatchable, into producer WriteToSpanner/make batches/Filtering Batchable Mutations/ParDo(_BatchableFilterFn) [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.883Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteToSpanner/Writing to spanner into WriteToSpanner/make batches/ParDo(_BatchFn) [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.913Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteToSpanner/make batches/Making mutation groups into Create/Read [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.952Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteToSpanner/make batches/Filtering Batchable Mutations/ParDo(_BatchableFilterFn) into WriteToSpanner/make batches/Making mutation groups [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:07.984Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteToSpanner/make batches/ParDo(_BatchFn) into WriteToSpanner/make batches/Filtering Batchable Mutations/ParDo(_BatchableFilterFn) [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.026Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.062Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.099Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.136Z: JOB_MESSAGE_DEBUG: Assigning stage ids. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.255Z: JOB_MESSAGE_DEBUG: Executing wait step start6 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.324Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+WriteToSpanner/make batches/Making mutation groups+WriteToSpanner/make batches/Filtering Batchable Mutations/ParDo(_BatchableFilterFn)+WriteToSpanner/Writing to spanner+WriteToSpanner/make batches/ParDo(_BatchFn)+WriteToSpanner/Writing to spanner [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.369Z: JOB_MESSAGE_DEBUG: Starting worker pool setup. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:08.393Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-a... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:44.413Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:40:47.207Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s). [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:41:15.255Z: JOB_MESSAGE_DETAILED: Workers have started successfully. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:46:37.903Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+WriteToSpanner/make batches/Making mutation groups+WriteToSpanner/make batches/Filtering Batchable Mutations/ParDo(_BatchableFilterFn)+WriteToSpanner/Writing to spanner+WriteToSpanner/make batches/ParDo(_BatchFn)+WriteToSpanner/Writing to spanner [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:46:37.996Z: JOB_MESSAGE_DEBUG: Executing success step success4 [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:46:38.072Z: JOB_MESSAGE_DETAILED: Cleaning up. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:46:38.140Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:46:38.184Z: JOB_MESSAGE_BASIC: Stopping worker pool... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:47:11.873Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized worker pool from 1 to 0. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:47:11.927Z: JOB_MESSAGE_BASIC: Worker pool stopped. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:242 2022-07-27T09:47:11.953Z: JOB_MESSAGE_DEBUG: Tearing down pending resources... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:197 Job 2022-07-27_02_40_01-11545245067662907333 is in state JOB_STATE_DONE [33m=============================== warnings summary ===============================[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees. database_id=self.TEST_DATABASE)) apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees. max_batch_size_bytes=250)) -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> - [31m[1m==== 2 failed, 1 passed, 4 skipped, 2 warnings, 10 error in 924.00 seconds =====[0m > Task :sdks:python:test-suites:dataflow:py37:spannerioIT FAILED FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 73 * What went wrong: Execution failed for task ':sdks:python:test-suites:direct:py37:spannerioIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 165 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:spannerioIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 2h 26m 55s 227 actionable tasks: 164 executed, 57 from cache, 6 up-to-date Publishing build scan... https://gradle.com/s/zdsc2o37pglc4 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
