[
https://issues.apache.org/jira/browse/BEAM-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362053#comment-17362053
]
Brian Hulette commented on BEAM-12474:
--------------------------------------
> Or is there a reason for adding it to PayloadSerializer as opposed to
> PubsubIO directly?
The reason to do it via PayloadSerializer would be to avoid duplicating logic
when we need the same feature in other IOs that support parsing payloads (e.g.
Kafka), and when the IOs are created from other contexts (from a SQL
TableProvider, or through an expansion service for a multi-language pipeline).
I think doing it in PubsubIO directly is an acceptable approach in the
short-term, but I also think we need to do some refactoring to enable sharing
this logic in the medium- to long-term.
> Configure PubsubIO with dead-letter topic
> -----------------------------------------
>
> Key: BEAM-12474
> URL: https://issues.apache.org/jira/browse/BEAM-12474
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Zachary Houfek
> Priority: P2
>
> Based on discussion from [PR14971|https://github.com/apache/beam/pull/14971].
> Currently, PubsubIO throws a RuntimeException when it can't parse a payload.
> It would perhaps be helpful if users could configure PubsubIO.Read with a
> dead-letter topic so that parsing errors instead write to the topic rather
> than just throw an exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
