[
https://issues.apache.org/jira/browse/BEAM-12482?focusedWorklogId=700327&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-700327
]
ASF GitHub Bot logged work on BEAM-12482:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Dec/21 23:21
Start Date: 22/Dec/21 23:21
Worklog Time Spent: 10m
Work Description: MiguelAnzoWizeline commented on pull request #16303:
URL: https://github.com/apache/beam/pull/16303#issuecomment-999936981
R: @chamikaramj Hi, I'm working on a fix for
https://issues.apache.org/jira/browse/BEAM-12482 that I see you were commenting
on, could you give it a review or know who can do it? The PR is an attempt to
replicate the fix done in https://github.com/apache/beam/pull/14113 for python.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 700327)
Time Spent: 2h 40m (was: 2.5h)
> BigQueryIO failed to load data to temp table when withSchemaUpdateOptions is
> set.
> ---------------------------------------------------------------------------------
>
> Key: BEAM-12482
> URL: https://issues.apache.org/jira/browse/BEAM-12482
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Siyuan Chen
> Assignee: Miguel Anzo
> Priority: P2
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> `withSchemaUpdateOptions` is only supported with WRITE_APPEND or
> WRITE_TRUNCATE with paritioned table. This is noted in the documentation:
> [https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1965]
> When the temp files are spilt to multiple load jobs, the write disposition is
> set to WRITE_TRUNCATE for the temporary tables in favor of retries.
> [https://github.com/apache/beam/blob/a8a8b3ab1292ff408940ba4430c95bc754341589/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L219]
> And the loading would fail with error
> "Schema update options should only be specified with WRITE_APPEND
> disposition, or with WRITE_TRUNCATE disposition on a table partition."
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
