See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/1338/display/redirect?page=changes>
Changes: [Kiley Sok] Update beam-master version [noreply] Remove datastore intersphinx_dependency to fix docs job. (#25000) [noreply] Use CC-BY image for custom remote inference notebook (#25004) [noreply] Fix kafka performance test write failing due to server overwhelmed ------------------------------------------ [...truncated 10.79 MB...] apache_beam/examples/complete/game/hourly_team_score_it_test.py: 1 warning apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings apache_beam/io/gcp/bigquery_file_loads_test.py: 3 warnings apache_beam/io/gcp/bigquery_write_it_test.py: 3 warnings apache_beam/io/gcp/bigquery_test.py: 3 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1175: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME') apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead. table_ref = client.dataset(dataset_id).table(table_id) apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:47: FutureWarning: The default value of numeric_only in DataFrame.mean is deprecated. In a future version, it will default to False. In addition, specifying 'numeric_only=None' is deprecated. Select only valid columns or specify the value of numeric_only to silence this warning. return airline_df[at_top_airports].mean() apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays apache_beam/examples/dataframe/taxiride_it_test.py::TaxirideIT::test_aggregation apache_beam/examples/dataframe/taxiride_it_test.py::TaxirideIT::test_enrich <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/dataframe/io.py>:659: FutureWarning: WriteToFiles is experimental. return pcoll | fileio.WriteToFiles( apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays apache_beam/examples/dataframe/taxiride_it_test.py::TaxirideIT::test_aggregation apache_beam/examples/dataframe/taxiride_it_test.py::TaxirideIT::test_enrich <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/fileio.py>:591: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported p.options.view_as(GoogleCloudOptions).temp_location or apache_beam/io/gcp/bigquery_test.py::BigQueryStreamingInsertTransformIntegrationTests::test_multiple_destinations_transform <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported streaming = self.test_pipeline.options.view_as(StandardOptions).streaming apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:172: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead. beam.io.BigQuerySource(query=self.query, use_standard_sql=True))) apache_beam/ml/gcp/cloud_dlp_it_test.py::CloudDLPIT::test_deidentification <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental. | MaskDetectedDetails( apache_beam/ml/gcp/cloud_dlp_it_test.py::CloudDLPIT::test_inspection <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental. | InspectForDetails( apache_beam/ml/inference/sklearn_inference_it_test.py::SklearnInference::test_sklearn_regression <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/examples/inference/sklearn_japanese_housing_regression.py>:128: FutureWarning: SklearnModelHandlerPandas is experimental. No backwards-compatibility guarantees. model_loader = SklearnModelHandlerPandas( apache_beam/ml/inference/sklearn_inference_it_test.py::SklearnInference::test_sklearn_regression <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/dill/_dill.py>:472: FutureWarning: SklearnModelHandlerPandas is experimental. No backwards-compatibility guarantees. obj = StockUnpickler.load(self) apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:695: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead. beam.io.BigQuerySource(query=self.query, use_standard_sql=True))) apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:810: FutureWarning: ReadAllFromBigQuery is experimental. | beam.io.ReadAllFromBigQuery()) apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2658: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2659: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2672: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported options=pcoll.pipeline.options, -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-df-py39.xml> - [36m[1m=========================== short test summary info ============================[0m [31mFAILED[0m apache_beam/io/gcp/bigquery_read_it_test.py::[1mReadNewTypesTests::test_iobase_source[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: Traceback (most recent call last): File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 837, in apache_beam.runners.common.PerWindowInvoker.invoke_process File "apache_beam/runners/common.py", line 983, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 1898, in <lambda> wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)] File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/testing/util.py";,> line 191, in _equal raise BeamAssertException(msg) apache_beam.testing.util.BeamAssertException: Failed assert: [{'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}] == [{'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10.000000000'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10.000000000'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': 0.33, 'numeric': Decimal('10.000000000'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': Decimal('10.000000000'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}], unexpected elements [{'float': None, 'numeric': Decimal('10.000000000'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': 0.33, 'numeric': Decimal('10.000000000'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 287, in _execute response = task() File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 360, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 596, in do_instruction return getattr(self, request_type)( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 634, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1003, in process_bundle input_op_by_transform_id[element.transform_id].process_encoded( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 227, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 526, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 528, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 237, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1491, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 623, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1581, in apache_beam.runners.common._OutputHandler.handle_process_outputs File "apache_beam/runners/common.py", line 1694, in apache_beam.runners.common._OutputHandler._write_value_to_tag File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1491, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 623, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1581, in apache_beam.runners.common._OutputHandler.handle_process_outputs File "apache_beam/runners/common.py", line 1694, in apache_beam.runners.common._OutputHandler._write_value_to_tag File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1491, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 623, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1581, in apache_beam.runners.common._OutputHandler.handle_process_outputs File "apache_beam/runners/common.py", line 1694, in apache_beam.runners.common._OutputHandler._write_value_to_tag File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1507, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 837, in apache_beam.runners.common.PerWindowInvoker.invoke_process File "apache_beam/runners/common.py", line 983, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 1898, in <lambda> wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)] File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/testing/util.py";,> line 191, in _equal raise BeamAssertException(msg) apache_beam.testing.util.BeamAssertException: Failed assert: [{'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}] == [{'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10.000000000'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10.000000000'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': 0.33, 'numeric': Decimal('10.000000000'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': Decimal('10.000000000'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}], unexpected elements [{'float': None, 'numeric': Decimal('10.000000000'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'\xab\xac', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': 0.33, 'numeric': Decimal('10.000000000'), 'bytes': b'\xab\xac', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}] [while running 'assert_that/Match-ptransform-43'] [31mFAILED[0m apache_beam/io/gcp/bigquery_read_it_test.py::[1mReadNewTypesTests::test_native_source[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: Traceback (most recent call last): File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 837, in apache_beam.runners.common.PerWindowInvoker.invoke_process File "apache_beam/runners/common.py", line 983, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 1898, in <lambda> wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)] File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/testing/util.py";,> line 191, in _equal raise BeamAssertException(msg) apache_beam.testing.util.BeamAssertException: Failed assert: [{'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'q6w=', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}] == [{'numeric': Decimal('10'), 'float': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'datetime': '2018-12-31T12:44:31', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'timestamp': None, 'geo': None}, {'geo': 'POINT(30 10)', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None}, {'timestamp': '2018-12-31 12:44:31.744957 UTC', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'geo': None}, {'timestamp': '2018-12-31 12:44:31.744957 UTC', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'geo': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'datetime': '2018-12-31T12:44:31', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'timestamp': None, 'geo': None}, {'numeric': Decimal('10'), 'float': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'time': '23:59:59', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'time': '23:59:59', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'geo': 'POINT(30 10)', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'bytes': b'q6w=', 'float': None, 'numeric': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'bytes': b'q6w=', 'float': None, 'numeric': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'date': '3000-12-31', 'float': None, 'numeric': None, 'bytes': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'date': '3000-12-31', 'float': None, 'numeric': None, 'bytes': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}], unexpected elements [{'timestamp': '2018-12-31 12:44:31.744957 UTC', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'geo': None}, {'datetime': '2018-12-31T12:44:31', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'timestamp': None, 'geo': None}, {'numeric': Decimal('10'), 'float': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'time': '23:59:59', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'geo': 'POINT(30 10)', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'bytes': b'q6w=', 'float': None, 'numeric': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'date': '3000-12-31', 'float': None, 'numeric': None, 'bytes': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 287, in _execute response = task() File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 360, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 596, in do_instruction return getattr(self, request_type)( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 634, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1003, in process_bundle input_op_by_transform_id[element.transform_id].process_encoded( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 227, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 526, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 528, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 237, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1491, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 623, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1581, in apache_beam.runners.common._OutputHandler.handle_process_outputs File "apache_beam/runners/common.py", line 1694, in apache_beam.runners.common._OutputHandler._write_value_to_tag File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1491, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 623, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1581, in apache_beam.runners.common._OutputHandler.handle_process_outputs File "apache_beam/runners/common.py", line 1694, in apache_beam.runners.common._OutputHandler._write_value_to_tag File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1491, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 623, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1581, in apache_beam.runners.common._OutputHandler.handle_process_outputs File "apache_beam/runners/common.py", line 1694, in apache_beam.runners.common._OutputHandler._write_value_to_tag File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 907, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 908, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1419, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1507, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1417, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 837, in apache_beam.runners.common.PerWindowInvoker.invoke_process File "apache_beam/runners/common.py", line 983, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 1898, in <lambda> wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)] File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/testing/util.py";,> line 191, in _equal raise BeamAssertException(msg) apache_beam.testing.util.BeamAssertException: Failed assert: [{'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': Decimal('10'), 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': b'q6w=', 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': '3000-12-31', 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': '23:59:59', 'datetime': None, 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': '2018-12-31T12:44:31', 'timestamp': None, 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': None}, {'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': 'POINT(30 10)'}] == [{'numeric': Decimal('10'), 'float': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'datetime': '2018-12-31T12:44:31', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'timestamp': None, 'geo': None}, {'geo': 'POINT(30 10)', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None}, {'timestamp': '2018-12-31 12:44:31.744957 UTC', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'geo': None}, {'timestamp': '2018-12-31 12:44:31.744957 UTC', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'geo': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'datetime': '2018-12-31T12:44:31', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'timestamp': None, 'geo': None}, {'numeric': Decimal('10'), 'float': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'time': '23:59:59', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'time': '23:59:59', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'geo': 'POINT(30 10)', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'bytes': b'q6w=', 'float': None, 'numeric': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'bytes': b'q6w=', 'float': None, 'numeric': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'date': '3000-12-31', 'float': None, 'numeric': None, 'bytes': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'date': '3000-12-31', 'float': None, 'numeric': None, 'bytes': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}, {'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}], unexpected elements [{'timestamp': '2018-12-31 12:44:31.744957 UTC', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'geo': None}, {'datetime': '2018-12-31T12:44:31', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'timestamp': None, 'geo': None}, {'numeric': Decimal('10'), 'float': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'time': '23:59:59', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'geo': 'POINT(30 10)', 'float': None, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None}, {'float': 0.33, 'numeric': None, 'bytes': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'bytes': b'q6w=', 'float': None, 'numeric': None, 'date': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'date': '3000-12-31', 'float': None, 'numeric': None, 'bytes': None, 'time': None, 'datetime': None, 'timestamp': None, 'geo': None}, {'float': 0.33, 'numeric': Decimal('10'), 'bytes': b'q6w=', 'date': '3000-12-31', 'time': '23:59:59', 'datetime': '2018-12-31T12:44:31', 'timestamp': '2018-12-31 12:44:31.744957 UTC', 'geo': 'POINT(30 10)'}] [while running 'assert_that/Match-ptransform-43'] [31m===== [31m[1m2 failed[0m, [32m83 passed[0m, [33m21 skipped[0m, [33m257 warnings[0m[31m in 14415.05s (4:00:15)[0m[31m =====[0m > Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 111 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 4h 2m 23s 207 actionable tasks: 144 executed, 57 from cache, 6 up-to-date Publishing build scan... https://gradle.com/s/x2ttbrdcmxsh4 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
