[
https://issues.apache.org/jira/browse/BEAM-7463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884789#comment-16884789
]
Valentyn Tymofieiev commented on BEAM-7463:
-------------------------------------------
[~pabloem] Which runner did you use? Seeing this on apache-beam-jenkins-11:
{noformat}
14:21:16 > Task :sdks:python:directRunnerIT
14:21:16 test_big_query_legacy_sql
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... ok
14:21:16 test_big_query_new_types
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... FAIL
14:21:16 test_big_query_standard_sql
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... ok
14:21:16 test_big_query_standard_sql_kms_key_native
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... SKIP: This test doesn't work on DirectRunner.
14:21:19 test_streaming_data_only
(apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
14:21:19 test_streaming_with_attributes
(apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
14:21:20 test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ok
14:21:20
14:21:20 ======================================================================
14:21:20 FAIL: test_big_query_new_types
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
14:21:20 ----------------------------------------------------------------------
14:21:20 Traceback (most recent call last):
14:21:20 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_it_test.py",
line 211, in test_big_query_new_types
14:21:20 big_query_query_to_table_pipeline.run_bq_pipeline(options)
14:21:20 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py",
line 82, in run_bq_pipeline
14:21:20 result = p.run()
14:21:20 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/testing/test_pipeline.py",
line 107, in run
14:21:20 else test_runner_api))
14:21:20 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/pipeline.py",
line 406, in run
14:21:20 self._options).run(False)
14:21:20 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/pipeline.py",
line 419, in run
14:21:20 return self.runner.run_pipeline(self, self._options)
14:21:20 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/apache_beam/runners/direct/test_direct_runner.py",
line 51, in run_pipeline
14:21:20 hc_assert_that(self.result, pickler.loads(on_success_matcher))
14:21:20 AssertionError:
14:21:20 Expected: (Test pipeline expected terminated in state: DONE and
Expected checksum is 1631ca7060b89a01760c81874b988c46156e18b5)
14:21:20 but:
14:21:20
14:21:20 -------------------- >> begin captured logging << --------------------
14:21:20 root: DEBUG: Connecting using Google Application Default Credentials.
14:21:20 oauth2client.transport: INFO: Attempting refresh to obtain initial
access_token
14:21:20 root: INFO: Running pipeline with DirectRunner.
14:21:20 root: DEBUG: Connecting using Google Application Default Credentials.
14:21:20 oauth2client.transport: INFO: Attempting refresh to obtain initial
access_token
14:21:20 root: INFO: Using location u'US' from table <TableReference
14:21:20 datasetId: u'python_query_to_table_15626208616472'
14:21:20 projectId: u'apache-beam-testing'
14:21:20 tableId: u'python_new_types_table'> referenced by query SELECT bytes,
date, time FROM [python_query_to_table_15626208616472.python_new_types_table]
14:21:20 root: WARNING: Dataset
apache-beam-testing:temp_dataset_f7ed13498f964c5888d607bd200e0e62 does not
exist so we will create it as temporary with location=US
14:21:20 root: DEBUG: Connecting using Google Application Default Credentials.
14:21:20 root: DEBUG: Creating or getting table <TableReference
14:21:20 datasetId: 'python_query_to_table_15626208616472'
14:21:20 projectId: 'apache-beam-testing'
14:21:20 tableId: 'output_table'> with schema {'fields': [{'type': u'BYTES',
'name': u'bytes', 'mode': 'NULLABLE'}, {'type': u'DATE', 'name': u'date',
'mode': 'NULLABLE'}, {'type': u'TIME', 'name': u'time', 'mode': 'NULLABLE'}]}.
14:21:20 oauth2client.transport: INFO: Attempting refresh to obtain initial
access_token
14:21:20 root: DEBUG: Created the table with id output_table
14:21:20 root: INFO: Created table
apache-beam-testing.python_query_to_table_15626208616472.output_table with
schema <TableSchema
14:21:20 fields: [<TableFieldSchema
14:21:20 fields: []
14:21:20 mode: u'NULLABLE'
14:21:20 name: u'bytes'
14:21:20 type: u'BYTES'>, <TableFieldSchema
14:21:20 fields: []
14:21:20 mode: u'NULLABLE'
14:21:20 name: u'date'
14:21:20 type: u'DATE'>, <TableFieldSchema
14:21:20 fields: []
14:21:20 mode: u'NULLABLE'
14:21:20 name: u'time'
14:21:20 type: u'TIME'>]>. Result: <Table
14:21:20 creationTime: 1562620866768
14:21:20 etag: u'qVCmDaC3dDohfi+Eh7thdw=='
14:21:20 id:
u'apache-beam-testing:python_query_to_table_15626208616472.output_table'
14:21:20 kind: u'bigquery#table'
14:21:20 lastModifiedTime: 1562620866835
14:21:20 location: u'US'
14:21:20 numBytes: 0
14:21:20 numLongTermBytes: 0
14:21:20 numRows: 0
14:21:20 schema: <TableSchema
14:21:20 fields: [<TableFieldSchema
14:21:20 fields: []
14:21:20 mode: u'NULLABLE'
14:21:20 name: u'bytes'
14:21:20 type: u'BYTES'>, <TableFieldSchema
14:21:20 fields: []
14:21:20 mode: u'NULLABLE'
14:21:20 name: u'date'
14:21:20 type: u'DATE'>, <TableFieldSchema
14:21:20 fields: []
14:21:20 mode: u'NULLABLE'
14:21:20 name: u'time'
14:21:20 type: u'TIME'>]>
14:21:20 selfLink:
u'https://www.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/python_query_to_table_15626208616472/tables/output_table'
14:21:20 tableReference: <TableReference
14:21:20 datasetId: u'python_query_to_table_15626208616472'
14:21:20 projectId: u'apache-beam-testing'
14:21:20 tableId: u'output_table'>
14:21:20 type: u'TABLE'>.
14:21:20 root: DEBUG: Attempting to flush to all destinations. Total buffered: 4
14:21:20 root: DEBUG: Flushing data to
apache-beam-testing:python_query_to_table_15626208616472.output_table. Total 4
rows.
14:21:20 root: DEBUG: Passed: True. Errors are []
14:21:20 root: INFO: Start verify Bigquery data.
14:21:20 google.auth.transport._http_client: DEBUG: Making request: GET
http://169.254.169.254
14:21:20 google.auth.transport._http_client: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/project/project-id
14:21:20 urllib3.util.retry: DEBUG: Converted retries value: 3 ->
Retry(total=3, connect=None, read=None, redirect=None, status=None)
14:21:20 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
14:21:20 urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
metadata.google.internal:80
14:21:20 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1"
200 144
14:21:20 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token
14:21:20 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/[email protected]/token
HTTP/1.1" 200 176
14:21:20 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
www.googleapis.com:443
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST
/bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/queries/6882e326-23ab-4624-8d80-732b2bd6e108?location=US&maxResults=0
HTTP/1.1" 200 None
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/jobs/6882e326-23ab-4624-8d80-732b2bd6e108?location=US
HTTP/1.1" 200 None
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon90739891049f0700f371e74e94b02d37a805a270/data
HTTP/1.1" 200 None
14:21:20 root: INFO: Read from given query (SELECT bytes, date, time FROM
`python_query_to_table_15626208616472.output_table`;), total rows 0
14:21:20 root: INFO: Generate checksum: da39a3ee5e6b4b0d3255bfef95601890afd80709
14:21:20 root: INFO: Start verify Bigquery data.
14:21:20 google.auth.transport._http_client: DEBUG: Making request: GET
http://169.254.169.254
14:21:20 google.auth.transport._http_client: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/project/project-id
14:21:20 urllib3.util.retry: DEBUG: Converted retries value: 3 ->
Retry(total=3, connect=None, read=None, redirect=None, status=None)
14:21:20 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
14:21:20 urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
metadata.google.internal:80
14:21:20 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1"
200 144
14:21:20 google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token
14:21:20 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/[email protected]/token
HTTP/1.1" 200 176
14:21:20 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
www.googleapis.com:443
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST
/bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/queries/d08af240-f21f-4d3e-a4d6-123520bd483a?location=US&maxResults=0
HTTP/1.1" 200 None
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/jobs/d08af240-f21f-4d3e-a4d6-123520bd483a?location=US
HTTP/1.1" 200 None
14:21:20 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon99c42be2_6544_4f73_a75e_cef6d47d7c8e/data
HTTP/1.1" 200 None
14:21:20 root: INFO: Read from given query (SELECT bytes, date, time FROM
`python_query_to_table_15626208616472.output_table`;), total rows 4
14:21:20 root: INFO: Generate checksum: 1631ca7060b89a01760c81874b988c46156e18b5
14:21:20 --------------------- >> end captured logging << ---------------------
14:21:20
14:21:20 ----------------------------------------------------------------------
14:21:20 XML:
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify_PR/src/sdks/python/nosetests.xml
14:21:20 ----------------------------------------------------------------------
14:21:20 Ran 9 tests in 24.097s
14:21:20
14:21:20 FAILED (SKIP=1, failures=1)
14:21:20
14:21:20 > Task :sdks:python:directRunnerIT FAILED
{noformat}
> Bigquery IO ITs are flaky: incorrect checksum
> ---------------------------------------------
>
> Key: BEAM-7463
> URL: https://issues.apache.org/jira/browse/BEAM-7463
> Project: Beam
> Issue Type: Bug
> Components: io-python-gcp
> Reporter: Valentyn Tymofieiev
> Assignee: Pablo Estrada
> Priority: Major
> Labels: currently-failing
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> {noformat}
> 15:03:38 FAIL: test_big_query_new_types
> (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
> 15:03:38
> ----------------------------------------------------------------------
> 15:03:38 Traceback (most recent call last):
> 15:03:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_it_test.py",
> line 211, in test_big_query_new_types
> 15:03:38 big_query_query_to_table_pipeline.run_bq_pipeline(options)
> 15:03:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py",
> line 82, in run_bq_pipeline
> 15:03:38 result = p.run()
> 15:03:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/testing/test_pipeline.py",
> line 107, in run
> 15:03:38 else test_runner_api))
> 15:03:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/pipeline.py",
> line 406, in run
> 15:03:38 self._options).run(False)
> 15:03:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/pipeline.py",
> line 419, in run
> 15:03:38 return self.runner.run_pipeline(self, self._options)
> 15:03:38 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/direct/test_direct_runner.py",
> line 51, in run_pipeline
> 15:03:38 hc_assert_that(self.result, pickler.loads(on_success_matcher))
> 15:03:38 AssertionError:
> 15:03:38 Expected: (Test pipeline expected terminated in state: DONE and
> Expected checksum is 24de460c4d344a4b77ccc4cc1acb7b7ffc11a214)
> 15:03:38 but: Expected checksum is
> 24de460c4d344a4b77ccc4cc1acb7b7ffc11a214 Actual checksum is
> da39a3ee5e6b4b0d3255bfef95601890afd80709
> {noformat}
> [~Juta] could this be caused by changes to Bigquery matcher?
> https://github.com/apache/beam/pull/8621/files#diff-f1ec7e3a3e7e2e5082ddb7043954c108R134
>
> cc: [~pabloem] [~chamikara] [~apilloud]
> A recent postcommit run has BQ failures in other tests as well:
> https://builds.apache.org/job/beam_PostCommit_Python3_Verify/1000/consoleFull
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
