[
https://issues.apache.org/jira/browse/BEAM-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Peterson resolved BEAM-1826.
----------------------------------
Resolution: Duplicate
Fix Version/s: Not applicable
> Allow BigqueryIO to forward errors
> ----------------------------------
>
> Key: BEAM-1826
> URL: https://issues.apache.org/jira/browse/BEAM-1826
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-gcp
> Reporter: Kevin Peterson
> Assignee: Daniel Halperin
> Priority: Minor
> Fix For: Not applicable
>
>
> Most sinks are terminal - data ends at the sink. While on occasion the sink
> may temporarily fail due to resource unavailability, it will eventually
> succeed. However, some have strict requirements on this input format. In
> these cases, retries will never succeed, and continuous retrying will
> eventually lead to pipeline failure.
> The primary use case I have in mind is streaming data to a sink such as
> BigQuery, where data of the wrong format could fail on insert.
> It would be useful to be able to set a side output or downstream transform
> from Bigquery which can receive failed rows where retry will never fix the
> issue, and allow them to be persisted to a different output which is more
> permissive of the output, to prevent data loss.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
