[
https://issues.apache.org/jira/browse/BEAM-8100?focusedWorklogId=307987&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307987
]
ASF GitHub Bot logged work on BEAM-8100:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Sep/19 16:08
Start Date: 06/Sep/19 16:08
Worklog Time Spent: 10m
Work Description: jklukas commented on pull request #9499: [BEAM-8100]
Exception handling for AsJsons and ParseJsons
URL: https://github.com/apache/beam/pull/9499#discussion_r321806969
##########
File path:
sdks/java/extensions/jackson/src/main/java/org/apache/beam/sdk/extensions/jackson/AsJsons.java
##########
@@ -56,6 +63,35 @@ private AsJsons(Class<? extends InputT> outputClass) {
return newTransform;
}
+ /**
+ * Returns a new {@link AsJsonsWithFailures} transform that catches
exceptions raised while
+ * writing JSON elements, passing the raised exception instance and the
input element being
+ * processed through the given {@code exceptionHandler} and emitting the
result to a failure
+ * collection.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * WithFailures.Result<PCollection<String>, KV<MyPojo, Map<String, String>>>
result =
+ * pojos.apply(
+ * AsJsons.of(MyPojo.class)
+ * .withFailures(new
WithFailures.ExceptionAsMapHandler<MyPojo>() {}));
+ *
+ * PCollection<String> output = result.output(); // valid json elements
+ * PCollection<KV<MyPojo, Map<String, String>>> failures = result.failures();
+ * }</pre>
+ */
+ @Experimental(Experimental.Kind.WITH_EXCEPTIONS)
+ public <FailureT> AsJsonsWithFailures<FailureT> withFailures(
Review comment:
> Agree to use the same API names for all transforms. Perhaps, it would make
sense to add an interface for that?
Yes, if it doesn't end cluttering the API too much. We could potentially put
this at `WithFailures.ExceptionsVia` and it would have to be templated on
`<InputT, FailureT, TransformT>` so we would end up with something like:
implements WithFailures.ExceptionsVia<String, FailureT,
AsJsonsWithFailures<FailureT>>
> Also, do we need to have separate method `exceptionsInto` only for lambdas?
Yes, I think we need a single `exceptionsInto` method. If the handler is
passed as a lambda, Beam likely won't be able to infer a coder for the failure
output collection without explicitly giving type information via
`exceptionsInto`.
> And finally, yes, I also was thinking to have default exception handler.
So, I think it will be a good idea to add this, thanks.
👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 307987)
Time Spent: 1h 40m (was: 1.5h)
> Add exception handling to Json transforms in Java SDK
> -----------------------------------------------------
>
> Key: BEAM-8100
> URL: https://issues.apache.org/jira/browse/BEAM-8100
> Project: Beam
> Issue Type: Improvement
> Components: extensions-java-json, sdk-java-core
> Reporter: Jeff Klukas
> Assignee: Alexey Romanenko
> Priority: Minor
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Follow-up to https://issues.apache.org/jira/browse/BEAM-5638. We have
> exception handling for MapElements and FlatMapElements. It should be
> straightforward to add parallel signatures to AsJsons and ParseJsons for
> catching parsing errors.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
