[ https://issues.apache.org/jira/browse/BEAM-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Cwik reassigned BEAM-4835: ------------------------------- Assignee: Gene Peters (was: Chamikara Jayalath) > Add more flexible options for data loading to BigQueryIO.Write > -------------------------------------------------------------- > > Key: BEAM-4835 > URL: https://issues.apache.org/jira/browse/BEAM-4835 > Project: Beam > Issue Type: Improvement > Components: io-java-gcp > Reporter: Gene Peters > Assignee: Gene Peters > Priority: Minor > Labels: pull-request-available > Fix For: 2.7.0 > > Time Spent: 50m > Remaining Estimate: 0h > > As part of the BigQuery API, there are a few options exposed to end-users > which allow for more flexible data loading. > For both > [streaming|https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/TableDataInsertAllRequest.html#setIgnoreUnknownValues-java.lang.Boolean-] > and > [batch|https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/JobConfigurationLoad.html#setIgnoreUnknownValues-java.lang.Boolean-] > inserts, the flag "ignoreUnknownValues" can be set, which indicates if > BigQuery should accept rows that contain values that do not match the schema. > [In > addition,|https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/TableDataInsertAllRequest.html#setSkipInvalidRows-java.lang.Boolean-] > streaming inserts allow for the option of accepting an inserted batch of > rows even if some of of the rows are invalid. > I've made the necessary code changes to make this available within > BigQueryIO.Write and will be attaching the pull request to this ticket for > review. Both flags are off by default. > Let me know if you have any questions or feedback about this! -- This message was sent by Atlassian JIRA (v7.6.3#76005)