[
https://issues.apache.org/jira/browse/BEAM-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-7396:
-------------------------------
Status: Open (was: Triage Needed)
> apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT.test_big_query_new_types
> is flaky
> ------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-7396
> URL: https://issues.apache.org/jira/browse/BEAM-7396
> Project: Beam
> Issue Type: Bug
> Components: io-python-gcp, test-failures
> Reporter: Valentyn Tymofieiev
> Assignee: Juta Staes
> Priority: Major
>
> cc: [~pabloem] [~chamikara] [~altay]
> {noformat}
> 16:23:37 test_big_query_new_types
> (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
> ... FAIL
> 16:23:38
> ======================================================================
> 16:23:38 FAIL: test_big_query_new_types
> (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
> 16:23:38
> ----------------------------------------------------------------------
> 16:23:38 Traceback (most recent call last):
> 16:23:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_it_test.py",
> line 217, in test_big_query_new_types
> 16:23:38 big_query_query_to_table_pipeline.run_bq_pipeline(options)
> 16:23:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py",
> line 82, in run_bq_pipeline
> 16:23:38 result = p.run()
> 16:23:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/testing/test_pipeline.py",
> line 107, in run
> 16:23:38 else test_runner_api))
> 16:23:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/pipeline.py",
> line 406, in run
> 16:23:38 self._options).run(False)
> 16:23:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/pipeline.py",
> line 419, in run
> 16:23:38 return self.runner.run_pipeline(self, self._options)
> 16:23:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/runners/direct/test_direct_runner.py",
> line 51, in run_pipeline
> 16:23:38 hc_assert_that(self.result, pickler.loads(on_success_matcher))
> 16:23:38 AssertionError:
> 16:23:38 Expected: (Test pipeline expected terminated in state: DONE and
> Expected checksum is 1631ca7060b89a01760c81874b988c46156e18b5)
> 16:23:38 but:
> 16:23:38
> 16:23:38 -------------------- >> begin captured logging <<
> --------------------
> 16:23:38 root: DEBUG: Connecting using Google Application Default Credentials.
> 16:23:38 oauth2client.transport: INFO: Attempting refresh to obtain initial
> access_token
> 16:23:38 root: INFO: Running pipeline with DirectRunner.
> 16:23:38 root: DEBUG: Connecting using Google Application Default Credentials.
> 16:23:38 oauth2client.transport: INFO: Attempting refresh to obtain initial
> access_token
> 16:23:38 root: INFO: Using location u'US' from table <TableReference
> 16:23:38 datasetId: u'python_query_to_table_1558567400576'
> 16:23:38 projectId: u'apache-beam-testing'
> 16:23:38 tableId: u'python_new_types_table'> referenced by query SELECT
> bytes, date, time FROM
> [python_query_to_table_1558567400576.python_new_types_table]
> 16:23:38 root: WARNING: Dataset
> apache-beam-testing:temp_dataset_d291c6c6fe2942e999d849ae5f2539ec does not
> exist so we will create it as temporary with location=US
> 16:23:38 root: DEBUG: Connecting using Google Application Default Credentials.
> 16:23:38 root: DEBUG: Creating or getting table <TableReference
> 16:23:38 datasetId: 'python_query_to_table_1558567400576'
> 16:23:38 projectId: 'apache-beam-testing'
> 16:23:38 tableId: 'output_table'> with schema {'fields': [{'type': u'BYTES',
> 'name': u'bytes', 'mode': 'NULLABLE'}, {'type': u'DATE', 'name': u'date',
> 'mode': 'NULLABLE'}, {'type': u'TIME', 'name': u'time', 'mode': 'NULLABLE'}]}.
> 16:23:38 oauth2client.transport: INFO: Attempting refresh to obtain initial
> access_token
> 16:23:38 root: DEBUG: Created the table with id output_table
> 16:23:38 root: INFO: Created table
> apache-beam-testing.python_query_to_table_1558567400576.output_table with
> schema <TableSchema
> 16:23:38 fields: [<TableFieldSchema
> 16:23:38 fields: []
> 16:23:38 mode: u'NULLABLE'
> 16:23:38 name: u'bytes'
> 16:23:38 type: u'BYTES'>, <TableFieldSchema
> 16:23:38 fields: []
> 16:23:38 mode: u'NULLABLE'
> 16:23:38 name: u'date'
> 16:23:38 type: u'DATE'>, <TableFieldSchema
> 16:23:38 fields: []
> 16:23:38 mode: u'NULLABLE'
> 16:23:38 name: u'time'
> 16:23:38 type: u'TIME'>]>. Result: <Table
> 16:23:38 creationTime: 1558567406362
> 16:23:38 etag: u'dzhADsNf4EAgsoVbnsI8/A=='
> 16:23:38 id:
> u'apache-beam-testing:python_query_to_table_1558567400576.output_table'
> 16:23:38 kind: u'bigquery#table'
> 16:23:38 lastModifiedTime: 1558567406435
> 16:23:38 location: u'US'
> 16:23:38 numBytes: 0
> 16:23:38 numLongTermBytes: 0
> 16:23:38 numRows: 0
> 16:23:38 schema: <TableSchema
> 16:23:38 fields: [<TableFieldSchema
> 16:23:38 fields: []
> 16:23:38 mode: u'NULLABLE'
> 16:23:38 name: u'bytes'
> 16:23:38 type: u'BYTES'>, <TableFieldSchema
> 16:23:38 fields: []
> 16:23:38 mode: u'NULLABLE'
> 16:23:38 name: u'date'
> 16:23:38 type: u'DATE'>, <TableFieldSchema
> 16:23:38 fields: []
> 16:23:38 mode: u'NULLABLE'
> 16:23:38 name: u'time'
> 16:23:38 type: u'TIME'>]>
> 16:23:38 selfLink:
> u'https://www.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/python_query_to_table_1558567400576/tables/output_table'
> 16:23:38 tableReference: <TableReference
> 16:23:38 datasetId: u'python_query_to_table_1558567400576'
> 16:23:38 projectId: u'apache-beam-testing'
> 16:23:38 tableId: u'output_table'>
> 16:23:38 type: u'TABLE'>.
> 16:23:38 root: DEBUG: Attempting to flush to all destinations. Total
> buffered: 4
> 16:23:38 root: DEBUG: Flushing data to
> apache-beam-testing:python_query_to_table_1558567400576.output_table. Total 4
> rows.
> 16:23:38 root: DEBUG: Passed: True. Errors are []
> 16:23:38 root: INFO: Start verify Bigquery data.
> 16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
> http://169.254.169.254
> 16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
> http://metadata.google.internal/computeMetadata/v1/project/project-id
> 16:23:38 urllib3.util.retry: DEBUG: Converted retries value: 3 ->
> Retry(total=3, connect=None, read=None, redirect=None, status=None)
> 16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
> http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
> 16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
> metadata.google.internal:80
> 16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80
> "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true
> HTTP/1.1" 200 144
> 16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
> http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token
> 16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80
> "GET
> /computeMetadata/v1/instance/service-accounts/[email protected]/token
> HTTP/1.1" 200 176
> 16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
> www.googleapis.com:443
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST
> /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
> /bigquery/v2/projects/apache-beam-testing/queries/f017abe1-973c-4f9e-807a-deeb8d5302cf?location=US&maxResults=0
> HTTP/1.1" 200 None
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
> /bigquery/v2/projects/apache-beam-testing/jobs/f017abe1-973c-4f9e-807a-deeb8d5302cf?location=US
> HTTP/1.1" 200 None
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
> /bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon14acc2c090fc251e1588e25913b043453f5b4d5e/data
> HTTP/1.1" 200 None
> 16:23:38 root: INFO: Read from given query (SELECT bytes, date, time FROM
> `python_query_to_table_1558567400576.output_table`;), total rows 0
> 16:23:38 root: INFO: Generate checksum:
> da39a3ee5e6b4b0d3255bfef95601890afd80709
> 16:23:38 root: INFO: Start verify Bigquery data.
> 16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
> http://169.254.169.254
> 16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
> http://metadata.google.internal/computeMetadata/v1/project/project-id
> 16:23:38 urllib3.util.retry: DEBUG: Converted retries value: 3 ->
> Retry(total=3, connect=None, read=None, redirect=None, status=None)
> 16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
> http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
> 16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
> metadata.google.internal:80
> 16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80
> "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true
> HTTP/1.1" 200 144
> 16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
> http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token
> 16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80
> "GET
> /computeMetadata/v1/instance/service-accounts/[email protected]/token
> HTTP/1.1" 200 176
> 16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
> www.googleapis.com:443
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST
> /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
> /bigquery/v2/projects/apache-beam-testing/queries/9d4f24bf-d44f-4823-b837-d01edf763159?location=US&maxResults=0
> HTTP/1.1" 200 None
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
> /bigquery/v2/projects/apache-beam-testing/jobs/9d4f24bf-d44f-4823-b837-d01edf763159?location=US
> HTTP/1.1" 200 None
> 16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
> /bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon6e2483c2_d7c7_4e33_9776_573d923304b7/data
> HTTP/1.1" 200 None
> 16:23:38 root: INFO: Read from given query (SELECT bytes, date, time FROM
> `python_query_to_table_1558567400576.output_table`;), total rows 4
> 16:23:38 root: INFO: Generate checksum:
> 1631ca7060b89a01760c81874b988c46156e18b5
> 16:23:38 --------------------- >> end captured logging <<
> ---------------------
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
