[
https://issues.apache.org/jira/browse/BEAM-8485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051190#comment-17051190
]
Christopher Dillon commented on BEAM-8485:
------------------------------------------
Just to add a little detail, seeing a similar issue, and looking at the
timestamps, it looks like the BQ job happens after the python pipeline
wait_until_finish returns successfully.
> Python Beam job is successful regardless of the result of the BigQuery load
> job (for the BigQuery File Loads)
> -------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8485
> URL: https://issues.apache.org/jira/browse/BEAM-8485
> Project: Beam
> Issue Type: Bug
> Components: io-py-gcp, sdk-py-core
> Affects Versions: 2.16.0
> Reporter: Robbie Gruener
> Priority: Major
>
> We are using the beam python sdk 2.16.0 and have noticed since upgrading to
> 2.16.0 that beam jobs are exiting successfully even though the BQ load job
> fails. We can see the load job failing in the Bigquery UI while the beam jobs
> running on either the DirectRunner or the DataflowRunner were successful.
> Note we are using the BigQuery File Loads (which is enabled via
> use_beam_bq_sink)
> I have verified that downgrading to 2.15.0 fixes this issue so it must be a
> newly introduced bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
