Mikhail Gryzykhin created BEAM-7410:
---------------------------------------
Summary: CLONE -
apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT.test_big_query_new_types
is flaky
Key: BEAM-7410
URL: https://issues.apache.org/jira/browse/BEAM-7410
Project: Beam
Issue Type: Bug
Components: io-python-gcp, test-failures
Reporter: Mikhail Gryzykhin
cc: [~pabloem] [~chamikara] [~altay]
{noformat}
16:23:37 test_big_query_new_types
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... FAIL
16:23:38 ======================================================================
16:23:38 FAIL: test_big_query_new_types
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
16:23:38 ----------------------------------------------------------------------
16:23:38 Traceback (most recent call last):
16:23:38 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_it_test.py",
line 217, in test_big_query_new_types
16:23:38 big_query_query_to_table_pipeline.run_bq_pipeline(options)
16:23:38 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py",
line 82, in run_bq_pipeline
16:23:38 result = p.run()
16:23:38 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/testing/test_pipeline.py",
line 107, in run
16:23:38 else test_runner_api))
16:23:38 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/pipeline.py",
line 406, in run
16:23:38 self._options).run(False)
16:23:38 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/pipeline.py",
line 419, in run
16:23:38 return self.runner.run_pipeline(self, self._options)
16:23:38 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/runners/direct/test_direct_runner.py",
line 51, in run_pipeline
16:23:38 hc_assert_that(self.result, pickler.loads(on_success_matcher))
16:23:38 AssertionError:
16:23:38 Expected: (Test pipeline expected terminated in state: DONE and
Expected checksum is 1631ca7060b89a01760c81874b988c46156e18b5)
16:23:38 but:
16:23:38
16:23:38 -------------------- >> begin captured logging << --------------------
16:23:38 root: DEBUG: Connecting using Google Application Default Credentials.
16:23:38 oauth2client.transport: INFO: Attempting refresh to obtain initial
access_token
16:23:38 root: INFO: Running pipeline with DirectRunner.
16:23:38 root: DEBUG: Connecting using Google Application Default Credentials.
16:23:38 oauth2client.transport: INFO: Attempting refresh to obtain initial
access_token
16:23:38 root: INFO: Using location u'US' from table <TableReference
16:23:38 datasetId: u'python_query_to_table_1558567400576'
16:23:38 projectId: u'apache-beam-testing'
16:23:38 tableId: u'python_new_types_table'> referenced by query SELECT bytes,
date, time FROM [python_query_to_table_1558567400576.python_new_types_table]
16:23:38 root: WARNING: Dataset
apache-beam-testing:temp_dataset_d291c6c6fe2942e999d849ae5f2539ec does not
exist so we will create it as temporary with location=US
16:23:38 root: DEBUG: Connecting using Google Application Default Credentials.
16:23:38 root: DEBUG: Creating or getting table <TableReference
16:23:38 datasetId: 'python_query_to_table_1558567400576'
16:23:38 projectId: 'apache-beam-testing'
16:23:38 tableId: 'output_table'> with schema {'fields': [{'type': u'BYTES',
'name': u'bytes', 'mode': 'NULLABLE'}, {'type': u'DATE', 'name': u'date',
'mode': 'NULLABLE'}, {'type': u'TIME', 'name': u'time', 'mode': 'NULLABLE'}]}.
16:23:38 oauth2client.transport: INFO: Attempting refresh to obtain initial
access_token
16:23:38 root: DEBUG: Created the table with id output_table
16:23:38 root: INFO: Created table
apache-beam-testing.python_query_to_table_1558567400576.output_table with
schema <TableSchema
16:23:38 fields: [<TableFieldSchema
16:23:38 fields: []
16:23:38 mode: u'NULLABLE'
16:23:38 name: u'bytes'
16:23:38 type: u'BYTES'>, <TableFieldSchema
16:23:38 fields: []
16:23:38 mode: u'NULLABLE'
16:23:38 name: u'date'
16:23:38 type: u'DATE'>, <TableFieldSchema
16:23:38 fields: []
16:23:38 mode: u'NULLABLE'
16:23:38 name: u'time'
16:23:38 type: u'TIME'>]>. Result: <Table
16:23:38 creationTime: 1558567406362
16:23:38 etag: u'dzhADsNf4EAgsoVbnsI8/A=='
16:23:38 id:
u'apache-beam-testing:python_query_to_table_1558567400576.output_table'
16:23:38 kind: u'bigquery#table'
16:23:38 lastModifiedTime: 1558567406435
16:23:38 location: u'US'
16:23:38 numBytes: 0
16:23:38 numLongTermBytes: 0
16:23:38 numRows: 0
16:23:38 schema: <TableSchema
16:23:38 fields: [<TableFieldSchema
16:23:38 fields: []
16:23:38 mode: u'NULLABLE'
16:23:38 name: u'bytes'
16:23:38 type: u'BYTES'>, <TableFieldSchema
16:23:38 fields: []
16:23:38 mode: u'NULLABLE'
16:23:38 name: u'date'
16:23:38 type: u'DATE'>, <TableFieldSchema
16:23:38 fields: []
16:23:38 mode: u'NULLABLE'
16:23:38 name: u'time'
16:23:38 type: u'TIME'>]>
16:23:38 selfLink:
u'https://www.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/python_query_to_table_1558567400576/tables/output_table'
16:23:38 tableReference: <TableReference
16:23:38 datasetId: u'python_query_to_table_1558567400576'
16:23:38 projectId: u'apache-beam-testing'
16:23:38 tableId: u'output_table'>
16:23:38 type: u'TABLE'>.
16:23:38 root: DEBUG: Attempting to flush to all destinations. Total buffered: 4
16:23:38 root: DEBUG: Flushing data to
apache-beam-testing:python_query_to_table_1558567400576.output_table. Total 4
rows.
16:23:38 root: DEBUG: Passed: True. Errors are []
16:23:38 root: INFO: Start verify Bigquery data.
16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
http://169.254.169.254
16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/project/project-id
16:23:38 urllib3.util.retry: DEBUG: Converted retries value: 3 ->
Retry(total=3, connect=None, read=None, redirect=None, status=None)
16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
metadata.google.internal:80
16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1"
200 144
16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/844138762903-comp...@developer.gserviceaccount.com/token
16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/844138762903-comp...@developer.gserviceaccount.com/token
HTTP/1.1" 200 176
16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
www.googleapis.com:443
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST
/bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/queries/f017abe1-973c-4f9e-807a-deeb8d5302cf?location=US&maxResults=0
HTTP/1.1" 200 None
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/jobs/f017abe1-973c-4f9e-807a-deeb8d5302cf?location=US
HTTP/1.1" 200 None
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon14acc2c090fc251e1588e25913b043453f5b4d5e/data
HTTP/1.1" 200 None
16:23:38 root: INFO: Read from given query (SELECT bytes, date, time FROM
`python_query_to_table_1558567400576.output_table`;), total rows 0
16:23:38 root: INFO: Generate checksum: da39a3ee5e6b4b0d3255bfef95601890afd80709
16:23:38 root: INFO: Start verify Bigquery data.
16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
http://169.254.169.254
16:23:38 google.auth.transport._http_client: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/project/project-id
16:23:38 urllib3.util.retry: DEBUG: Converted retries value: 3 ->
Retry(total=3, connect=None, read=None, redirect=None, status=None)
16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
metadata.google.internal:80
16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1"
200 144
16:23:38 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/844138762903-comp...@developer.gserviceaccount.com/token
16:23:38 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/844138762903-comp...@developer.gserviceaccount.com/token
HTTP/1.1" 200 176
16:23:38 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
www.googleapis.com:443
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST
/bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/queries/9d4f24bf-d44f-4823-b837-d01edf763159?location=US&maxResults=0
HTTP/1.1" 200 None
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/jobs/9d4f24bf-d44f-4823-b837-d01edf763159?location=US
HTTP/1.1" 200 None
16:23:38 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon6e2483c2_d7c7_4e33_9776_573d923304b7/data
HTTP/1.1" 200 None
16:23:38 root: INFO: Read from given query (SELECT bytes, date, time FROM
`python_query_to_table_1558567400576.output_table`;), total rows 4
16:23:38 root: INFO: Generate checksum: 1631ca7060b89a01760c81874b988c46156e18b5
16:23:38 --------------------- >> end captured logging << ---------------------
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
