[
https://issues.apache.org/jira/browse/BEAM-14447?focusedWorklogId=768602&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768602
]
ASF GitHub Bot logged work on BEAM-14447:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/May/22 17:12
Start Date: 10/May/22 17:12
Worklog Time Spent: 10m
Work Description: pabloem commented on PR #17601:
URL: https://github.com/apache/beam/pull/17601#issuecomment-1122659567
Run Python 3.8 PostCommit
Issue Time Tracking
-------------------
Worklog Id: (was: 768602)
Time Spent: 0.5h (was: 20m)
> BigQueryWriteIntegrationTests.test_big_query_write_insert_errors_reporting
> failing in Python PostCommit
> -------------------------------------------------------------------------------------------------------
>
> Key: BEAM-14447
> URL: https://issues.apache.org/jira/browse/BEAM-14447
> Project: Beam
> Issue Type: Bug
> Components: io-py-gcp, test-failures
> Reporter: Brian Hulette
> Priority: P1
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> A new test added in [https://github.com/apache/beam/pull/17517] is failing at
> HEAD.
>
> [https://ci-beam.apache.org/job/beam_PostCommit_Python37/5198/testReport/junit/apache_beam.io.gcp.bigquery_write_it_test/BigQueryWriteIntegrationTests/test_big_query_write_insert_errors_reporting/]
>
> {code:java}
> pipeline_verifiers = [
> BigqueryFullResultMatcher(
> project=self.project,
> query="SELECT number, str FROM %s" % table_id,
> data=[(1, 'some_string')]),
> BigqueryFullResultMatcher(
> project=self.project,
> query="SELECT table, reason, row_json FROM %s" %
> errors_table_id,
> data=
> [(
> table_id,
> '[{"reason": "invalid", "location": "", "debugInfo": "", \
> "message": "Missing required field: Msg_0_CLOUD_QUERY_TABLE.str."}]',
> '{"number": 2}'),
> (
> table_id,
> '[{"reason": "invalid", "location":
> "additional_field_str", \
> "debugInfo": "", "message": "no such field: additional_field_str."}]',
> '{"number": 3, "str": "some_string",
> "additional_field_str": \
> "some_string"}')])
> ]
>
> args = self.test_pipeline.get_full_options_as_args(
> on_success_matcher=hc.all_of(*pipeline_verifiers))
>
> with beam.Pipeline(argv=args) as p:
> # pylint: disable=expression-not-assigned
> errors = (
> p | 'create' >> beam.Create(input_data)
> | 'write' >> beam.io.WriteToBigQuery(
> table_id,
> schema=table_schema,
>
> create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED,
> write_disposition=beam.io.BigQueryDisposition.WRITE_EMPTY))
> (
> errors["FailedRows"]
> | 'ParseErrors' >> beam.Map(
> lambda err: {
> "table": err[0],
> "reason": json.dumps(err[2]),
> "row_json": json.dumps(err[1])
> })
> > | 'WriteErrors' >> beam.io.WriteToBigQuery(
> errors_table_id,
> schema=errors_table_schema,
>
> create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED,
> write_disposition=beam.io.BigQueryDisposition.WRITE_EMPTY))
> E KeyError: 'FailedRows'
> apache_beam/io/gcp/bigquery_write_it_test.py:461: KeyError {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
