See <https://builds.apache.org/job/beam_PostCommit_Python36/1768/display/redirect?page=changes>
Changes: [huangry] Create validation runner test for metrics (limited to user counter in [huangry] Fixups. [heejong] [BEAM-9341] postcommit xvr flink fix [github] [BEAM-8280] Enable and improve IOTypeHints debug_str traceback (#10894) ------------------------------------------ [...truncated 2.60 MB...] test_iobase_source (apache_beam.io.gcp.bigquery_read_it_test.ReadTests) ... ok test_native_source (apache_beam.io.gcp.bigquery_read_it_test.ReadTests) ... ok test_bqfl_streaming (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... SKIP: TestStream is not supported on TestDataflowRunner test_multiple_destinations_transform (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... ok test_one_job_fails_all_jobs_fail (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... ok test_copy (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok test_copy_batch (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok test_copy_batch_kms (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok test_copy_batch_rewrite_token (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... FAIL test_copy_kms (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok test_copy_rewrite_token (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok test_transform_on_gcs (apache_beam.io.fileio_test.MatchIntegrationTest) ... ok test_parquetio_it (apache_beam.io.parquetio_it_test.TestParquetIT) ... ok test_file_loads (apache_beam.io.gcp.bigquery_test.PubSubBigQueryIT) ... SKIP: https://issuetracker.google.com/issues/118375066 test_streaming_inserts (apache_beam.io.gcp.bigquery_test.PubSubBigQueryIT) ... ok test_basic_execution (apache_beam.testing.test_stream_it_test.TestStreamIntegrationTests) ... SKIP: The "TestDataflowRunner", does not support the TestStream transform. Supported runners: ['DirectRunner', 'SwitchingDirectRunner'] Tests that the TestStream supports emitting to multiple PCollections. ... SKIP: The "TestDataflowRunner", does not support the TestStream transform. Supported runners: ['DirectRunner', 'SwitchingDirectRunner'] Tests that the TestStream can independently control output watermarks. ... SKIP: The "TestDataflowRunner", does not support the TestStream transform. Supported runners: ['DirectRunner', 'SwitchingDirectRunner'] test_streaming_data_only (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok test_streaming_with_attributes (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok test_datastore_write_limit (apache_beam.io.gcp.datastore.v1new.datastore_write_it_test.DatastoreWriteIT) ... ok Runs streaming Dataflow job and verifies that user metrics are reported ... ok test_job_python_from_python_it (apache_beam.transforms.external_it_test.ExternalTransformIT) ... ok test_big_query_write (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok test_big_query_write_new_types (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok test_big_query_write_schema_autodetect (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... SKIP: DataflowRunner does not support schema autodetection test_big_query_write_without_schema (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok test_metrics_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok test_big_query_legacy_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok test_big_query_new_types (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok test_big_query_new_types_native (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok test_big_query_standard_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok test_big_query_standard_sql_kms_key_native (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok ====================================================================== FAIL: test_copy_batch_rewrite_token (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/gcsio_integration_test.py";,> line 196, in test_copy_batch_rewrite_token src=self.INPUT_FILE_LARGE) File "<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/gcsio_integration_test.py";,> line 165, in _test_copy_batch self.assertIsNone(status[2], status) AssertionError: HttpError('HttpError accessing <https://www.googleapis.com/batch/storage/v1>: response: <{\'Content-Type\': \'application/json; charset=UTF-8\', \'Date\': \'Thu, 20 Feb 2020 22:31:33 GMT\', \'Expires\': \'Thu, 20 Feb 2020 22:31:33 GMT\', \'Cache-Control\': \'private, max-age=0\', \'Content-Length\': \'177\', \'status\': \'503\'}>, content <{\n "error": {\n "errors": [\n {\n "domain": "global",\n "reason": "backendError",\n "message": "Backend Error"\n }\n ],\n "code": 503,\n "message": "Backend Error"\n }\n}\n>',) is not None : ('gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json', 'gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_9', HttpError('HttpError accessing <https://www.googleapis.com/batch/storage/v1>: response: <{\'Content-Type\': \'application/json; charset=UTF-8\', \'Date\': \'Thu, 20 Feb 2020 22:31:33 GMT\', \'Expires\': \'Thu, 20 Feb 2020 22:31:33 GMT\', \'Cache-Control\': \'private, max-age=0\', \'Content-Length\': \'177\', \'status\': \'503\'}>, content <{\n "error": {\n "errors": [\n {\n "domain": "global",\n "reason": "backendError",\n "message": "Backend Error"\n }\n ],\n "code": 503,\n "message": "Backend Error"\n }\n}\n>',)) -------------------- >> begin captured logging << -------------------- apache_beam.options.pipeline_options: WARNING: --region not set; will default to us-central1. Future releases of Beam will require the user to set --region explicitly, or else have a default set via the gcloud tool. https://cloud.google.com/compute/docs/regions-zones apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_0 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_2 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_9 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_6 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_4 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_7 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_3 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_8 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_1 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 52428800 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_5 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_0 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_2 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_9 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_6 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_4 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_7 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_3 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_8 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_1 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 104857600 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_5 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_0 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_2 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_9 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_6 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_4 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_7 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_3 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_8 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 113246208 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_1 apache_beam.io.gcp.gcsio: DEBUG: Rewrite progress: 157286400 of 161612163 bytes, gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_5 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_0 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_2 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_6 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_4 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_7 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_3 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_8 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_1 apache_beam.io.gcp.gcsio: DEBUG: Rewrite done: gs://dataflow-samples/wikipedia_edits/wiki_data-000000000000.json to gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/test_copy_batch_rewrite_token_5 apache_beam.io.filesystem: DEBUG: Listing files in 'gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/' apache_beam.io.filesystem: DEBUG: translate_pattern: 'gs://temp-storage-for-end-to-end-tests/temp-it/gcs_it-382c3e41-94f0-4865-ada9-3ab123b7c8ca/*' -> 'gs\\:\\/\\/temp\\-storage\\-for\\-end\\-to\\-end\\-tests\\/temp\\-it\\/gcs_it\\-382c3e41\\-94f0\\-4865\\-ada9\\-3ab123b7c8ca\\/[^/\\\\]*' apache_beam.io.gcp.gcsio: INFO: Starting the size estimation of the input apache_beam.io.gcp.gcsio: INFO: Finished listing 9 files in 0.05834603309631348 seconds. --------------------- >> end captured logging << --------------------- <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1466: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_14-10736950041723466208?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_14_44-14277888231934516734?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_21_47-1806567505907177665?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:275: FutureWarning: _ReadFromBigQuery is experimental. query=self.query, use_standard_sql=True, project=self.project)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_32_02-13257704391024410351?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1655: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_39_02-14492178607796888749?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_09-4539706728852269119?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_16_52-12811813950644362544?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=transform.kms_key)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_23_59-4760705086600559061?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:823: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported streaming = self.test_pipeline.options.view_as(StandardOptions).streaming <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1466: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1463: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self.table_reference.projectId = pcoll.pipeline.options.view_as( Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_11-17215498208795092095?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1466: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_12_10-7995678639820605625?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_18_49-5411070958172706084?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_26_32-11532403842998844119?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_32_45-5477159194795998415?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:87: FutureWarning: _ReadFromBigQuery is experimental. kms_key=kms_key) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_40_51-7392853184720235955?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1655: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=transform.kms_key)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:87: FutureWarning: _ReadFromBigQuery is experimental. kms_key=kms_key) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1655: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=transform.kms_key)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:95: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=kms_key)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:87: FutureWarning: _ReadFromBigQuery is experimental. kms_key=kms_key) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1655: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=transform.kms_key)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:95: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=kms_key)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_09-844689954589638892?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_15_56-9691643896236703237?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_22_41-15221771287090582189?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_29_18-8344408336196553011?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:162: FutureWarning: _ReadFromBigQuery is experimental. query=self.query, use_standard_sql=True, project=self.project)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_37_09-5982510871736479911?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1655: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio_test.py>:303: FutureWarning: MatchAll is experimental. | 'GetPath' >> beam.Map(lambda metadata: metadata.path)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio_test.py>:316: FutureWarning: MatchAll is experimental. | 'Checksums' >> beam.Map(compute_hash)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio_test.py>:316: FutureWarning: ReadMatches is experimental. | 'Checksums' >> beam.Map(compute_hash)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_08-16485762822951707116?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_io_read_pipeline.py>:84: FutureWarning: _ReadFromBigQuery is experimental. (options.view_as(GoogleCloudOptions).project, known_args.input_table)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_07_33-10339065497297810768?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1655: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_14_39-4750837915247016696?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_21_47-6912655895968443447?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_io_read_pipeline.py>:84: FutureWarning: _ReadFromBigQuery is experimental. Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_29_23-12998459365886416504?project=apache-beam-testing (options.view_as(GoogleCloudOptions).project, known_args.input_table)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_36_05-7517177145837539959?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1655: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_10-12347991921498489255?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_06_55-494537295153221784?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_14_12-316958759806868608?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_22_36-8853958681682436791?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_30_52-15309777217217950467?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_38_27-6142029644651186365?project=apache-beam-testing kms_key=transform.kms_key)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1466: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:818: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = p.options.view_as(GoogleCloudOptions).temp_location <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1466: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:818: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = p.options.view_as(GoogleCloudOptions).temp_location Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_09-5215021210097153082?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_08_34-10841802320858527209?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/avroio.py>:205: UserWarning: Due to a known issue in avro-python3 package, it is recommended to use fastavro with Beam Avro IO on Python 3 until BEAM-6522 is addressed. "Due to a known issue in avro-python3 package, it is " <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1466: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_17_57-3673669532468774557?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_25_12-9634047932736766013?project=apache-beam-testing <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:818: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_32_14-9659989138280072434?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_39_03-226062581047068695?project=apache-beam-testing temp_location = p.options.view_as(GoogleCloudOptions).temp_location <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=transform.kms_key)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=transform.kms_key)) <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:788: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=transform.kms_key)) Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_00_12-6849706905715557371?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_08_42-10074736828066067448?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_15_11-5072205975285174636?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_22_14-6687702944812274455?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_29_19-11145045011936077452?project=apache-beam-testing Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-20_14_36_13-15869353556100870590?project=apache-beam-testing ---------------------------------------------------------------------- XML: nosetests-postCommitIT-df-py36.xml ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 52 tests in 2866.745s FAILED (SKIP=8, failures=1) > Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/py36/build.gradle'> line: 56 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 48m 54s 84 actionable tasks: 63 executed, 21 from cache Publishing build scan... https://gradle.com/s/p3iefkptaqfzu Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
