[
https://issues.apache.org/jira/browse/BEAM-12474?focusedWorklogId=622687&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622687
]
ASF GitHub Bot logged work on BEAM-12474:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Jul/21 19:52
Start Date: 14/Jul/21 19:52
Worklog Time Spent: 10m
Work Description: kennknowles commented on a change in pull request
#15037:
URL: https://github.com/apache/beam/pull/15037#discussion_r669908020
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/WithFailures.java
##########
@@ -92,6 +93,24 @@
}
}
+ /**
+ * A handler that holds onto the {@link Throwable} that led to the
exception, returning it along
+ * with the original value as a {@link KV}.
+ *
+ * <p>Extends {@link SimpleFunction} so that full type information is
captured. {@link KV} and
+ * {@link ComparableThrowable} coders can be easily inferred by Beam, so
coder inference can be
+ * successfully applied if the consuming transform passes type information
to the failure
+ * collection's {@link TupleTag}. This may require creating an instance of
an anonymous inherited
+ * class rather than of this class directly.
+ */
+ public static class ThrowableHandler<T>
+ extends SimpleFunction<ExceptionElement<T>, KV<T, ComparableThrowable>> {
+ @Override
+ public KV<T, ComparableThrowable> apply(ExceptionElement<T> f) {
+ return KV.of(f.element(),
ComparableThrowable.forThrowable(f.exception()));
Review comment:
Pipeline update checks if all coders are the same. Technically only
coders used for persisted state in a shuffle. So if the binary encoding of the
coder changes, it will be rejected for update. It was my mistake - it is
actually `CustomCoder` that may have spurious incompatibilities. For
`SerializableCoder` there may be similar risk because it contains the
serialized `Class` object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 622687)
Time Spent: 5h 50m (was: 5h 40m)
> Configure PubsubIO with dead-letter topic
> -----------------------------------------
>
> Key: BEAM-12474
> URL: https://issues.apache.org/jira/browse/BEAM-12474
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Zachary Houfek
> Assignee: Zachary Houfek
> Priority: P2
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Based on discussion from [PR14971|https://github.com/apache/beam/pull/14971].
> Currently, PubsubIO throws a RuntimeException when it can't parse a payload.
> It would perhaps be helpful if users could configure PubsubIO.Read with a
> dead-letter topic so that parsing errors instead write to the topic rather
> than just throw an exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
