[
https://issues.apache.org/jira/browse/BEAM-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947374#comment-15947374
]
Daniel Halperin commented on BEAM-1826:
---------------------------------------
This sounds like [BEAM-190] to me -- is that accurate?
> Allow BigqueryIO to forward errors
> ----------------------------------
>
> Key: BEAM-1826
> URL: https://issues.apache.org/jira/browse/BEAM-1826
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-gcp
> Reporter: Kevin Peterson
> Assignee: Daniel Halperin
> Priority: Minor
>
> Most sinks are terminal - data ends at the sink. While on occasion the sink
> may temporarily fail due to resource unavailability, it will eventually
> succeed. However, some have strict requirements on this input format. In
> these cases, retries will never succeed, and continuous retrying will
> eventually lead to pipeline failure.
> The primary use case I have in mind is streaming data to a sink such as
> BigQuery, where data of the wrong format could fail on insert.
> It would be useful to be able to set a side output or downstream transform
> from Bigquery which can receive failed rows where retry will never fix the
> issue, and allow them to be persisted to a different output which is more
> permissive of the output, to prevent data loss.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
