See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2343/display/redirect>
Changes: ------------------------------------------ [...truncated 12.21 MB...] apache_beam/io/gcp/bigquery_io_read_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 3 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2785: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( apache_beam/dataframe/io_it_test.py: 3 warnings apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/examples/dataframe/flight_delays_it_test.py: 1 warning apache_beam/io/gcp/bigquery_read_it_test.py: 7 warnings apache_beam/io/gcp/bigquery_io_read_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 3 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2787: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name apache_beam/dataframe/io_it_test.py: 3 warnings apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/examples/dataframe/flight_delays_it_test.py: 1 warning apache_beam/io/gcp/bigquery_read_it_test.py: 7 warnings apache_beam/io/gcp/bigquery_io_read_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 3 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2811: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported pipeline_options=pcoll.pipeline.options, apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_file_loads apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_streaming_inserts <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead. dataset_ref = client.dataset(unique_dataset_name, project=project) apache_beam/examples/complete/game/hourly_team_score_it_test.py: 1 warning apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/examples/complete/game/game_stats_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_file_loads_test.py: 3 warnings apache_beam/examples/complete/game/leader_board_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_write_it_test.py: 5 warnings apache_beam/io/gcp/bigquery_test.py: 6 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 2 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2107: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported is_streaming_pipeline = p.options.view_as(StandardOptions).streaming apache_beam/examples/complete/game/hourly_team_score_it_test.py: 1 warning apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/examples/complete/game/game_stats_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_file_loads_test.py: 3 warnings apache_beam/examples/complete/game/leader_board_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_write_it_test.py: 5 warnings apache_beam/io/gcp/bigquery_test.py: 6 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 2 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2113: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] apache_beam/examples/complete/game/hourly_team_score_it_test.py: 1 warning apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/io/gcp/bigquery_file_loads_test.py: 3 warnings apache_beam/io/gcp/bigquery_write_it_test.py: 3 warnings apache_beam/io/gcp/bigquery_test.py: 3 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1183: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = p.options.view_as(GoogleCloudOptions).temp_location apache_beam/examples/complete/game/hourly_team_score_it_test.py: 1 warning apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/io/gcp/bigquery_file_loads_test.py: 3 warnings apache_beam/io/gcp/bigquery_write_it_test.py: 3 warnings apache_beam/io/gcp/bigquery_test.py: 3 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1185: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME') apache_beam/dataframe/io_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 1 warning apache_beam/io/gcp/bigquery_read_it_test.py: 10 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2829: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project apache_beam/dataframe/io_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 1 warning apache_beam/io/gcp/bigquery_read_it_test.py: 10 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2850: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported pipeline_options=pcoll.pipeline.options, apache_beam/dataframe/io_it_test.py: 2 warnings apache_beam/io/gcp/bigquery_json_it_test.py: 1 warning apache_beam/io/gcp/bigquery_read_it_test.py: 10 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_internal.py>:150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported pipeline_options = input.pipeline.options apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/io/gcp/bigquery_write_it_test.py: 5 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings apache_beam/io/gcp/bigquery_test.py: 2 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2103: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self.table_reference.projectId = pcoll.pipeline.options.view_as( apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py: 1 warning apache_beam/io/gcp/bigquery_file_loads_test.py: 3 warnings apache_beam/io/gcp/bigquery_write_it_test.py: 3 warnings apache_beam/io/gcp/bigquery_test.py: 3 warnings apache_beam/io/gcp/big_query_query_to_table_it_test.py: 4 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1176: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self.project = self.project or p.options.view_as(GoogleCloudOptions).project apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead. table_ref = client.dataset(dataset_id).table(table_id) apache_beam/examples/dataframe/taxiride_it_test.py::TaxirideIT::test_aggregation apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays apache_beam/examples/dataframe/taxiride_it_test.py::TaxirideIT::test_enrich <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/fileio.py>:605: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported p.options.view_as(GoogleCloudOptions).temp_location or apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:47: FutureWarning: The default value of numeric_only in DataFrame.mean is deprecated. In a future version, it will default to False. In addition, specifying 'numeric_only=None' is deprecated. Select only valid columns or specify the value of numeric_only to silence this warning. return airline_df[at_top_airports].mean() apache_beam/io/gcp/bigquery_test.py::BigQueryStreamingInsertTransformIntegrationTests::test_multiple_destinations_transform <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:2097: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported streaming = self.test_pipeline.options.view_as(StandardOptions).streaming apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:170: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead. beam.io.BigQuerySource(query=self.query, use_standard_sql=True))) apache_beam/transforms/external_it_test.py::ExternalTransformIT::test_job_python_from_python_it <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/transforms/external.py>:725: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self._expansion_service, pipeline.options) apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:706: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead. beam.io.BigQuerySource(query=self.query, use_standard_sql=True))) apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2957: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2958: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2971: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported options=pcoll.pipeline.options, -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-df-py39.xml> - [36m[1m=========================== short test summary info ============================[0m [31mFAILED[0m apache_beam/examples/complete/game/hourly_team_score_it_test.py::[1mHourlyTeamScoreIT::test_hourly_team_score_it[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 297, in _execute response = task() File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 372, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 625, in do_instruction return getattr(self, request_type)( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 656, in process_bundle bundle_processor = self.bundle_processor_cache.get( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 487, in get processor = bundle_processor.BundleProcessor( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 888, in __init__ _verify_descriptor_created_in_a_compatible_env(process_bundle_descriptor) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 851, in _verify_descriptor_created_in_a_compatible_env raise RuntimeError( RuntimeError: Pipeline construction environment and pipeline runtime environment are not compatible. If you use a custom container image, check that the Python interpreter minor version and the Apache Beam version in your image match the versions used at pipeline construction time. Submission environment: beam:version:sdk_base:apache/beam_python3.9_sdk:2.52.0.dev. Runtime environment: beam:version:sdk_base:apache/beam_python3.9_sdk:2.51.0.dev. [31mFAILED[0m apache_beam/examples/cookbook/datastore_wordcount_it_test.py::[1mDatastoreWordCountIT::test_datastore_wordcount_it[0m - apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Sat, 23 Sep 2023 00:42:06 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '491', '-content-encoding': 'gzip'}>, content <{ "error": { "code": 400, "message": "(a17963b6ea21c276): The workflow could not be created. Causes: (507a42656c5621f): Dataflow quota error for jobs-per-project quota. Project apache-beam-testing is running 302 jobs. Please check the quota usage via GCP Console. If it exceeds the limit, please wait for a workflow to finish or contact Google Cloud Support to request an increase in quota. If it does not, contact Google Cloud Support.", "status": "FAILED_PRECONDITION" } } > [31mFAILED[0m apache_beam/examples/dataframe/taxiride_it_test.py::[1mTaxirideIT::test_aggregation[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 297, in _execute response = task() File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 372, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 625, in do_instruction return getattr(self, request_type)( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 656, in process_bundle bundle_processor = self.bundle_processor_cache.get( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 487, in get processor = bundle_processor.BundleProcessor( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 888, in __init__ _verify_descriptor_created_in_a_compatible_env(process_bundle_descriptor) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 851, in _verify_descriptor_created_in_a_compatible_env raise RuntimeError( RuntimeError: Pipeline construction environment and pipeline runtime environment are not compatible. If you use a custom container image, check that the Python interpreter minor version and the Apache Beam version in your image match the versions used at pipeline construction time. Submission environment: beam:version:sdk_base:apache/beam_python3.9_sdk:2.52.0.dev. Runtime environment: beam:version:sdk_base:apache/beam_python3.9_sdk:2.51.0.dev. [31mFAILED[0m apache_beam/transforms/external_it_test.py::[1mExternalTransformIT::test_job_python_from_python_it[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 297, in _execute response = task() File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 372, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 625, in do_instruction return getattr(self, request_type)( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 656, in process_bundle bundle_processor = self.bundle_processor_cache.get( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 487, in get processor = bundle_processor.BundleProcessor( File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 888, in __init__ _verify_descriptor_created_in_a_compatible_env(process_bundle_descriptor) File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 851, in _verify_descriptor_created_in_a_compatible_env raise RuntimeError( RuntimeError: Pipeline construction environment and pipeline runtime environment are not compatible. If you use a custom container image, check that the Python interpreter minor version and the Apache Beam version in your image match the versions used at pipeline construction time. Submission environment: beam:version:sdk_base:apache/beam_python3.9_sdk:2.52.0.dev. Runtime environment: beam:version:sdk_base:apache/beam_python3.9_sdk:2.51.0.dev. [31m===== [31m[1m4 failed[0m, [32m85 passed[0m, [33m50 skipped[0m, [33m623 warnings[0m[31m in 7597.88s (2:06:37)[0m[31m ======[0m > Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 2h 13m 8s 214 actionable tasks: 149 executed, 59 from cache, 6 up-to-date Publishing build scan... https://ge.apache.org/s/xp5c2kf5bqke6 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
