liferoad commented on issue #31193: URL: https://github.com/apache/beam/issues/31193#issuecomment-2096737468
> @liferoad while that's helpful as a pattern to look at, I don't think that solves the general issue of catching exceptions in transforms for 2 reasons: > > 1. That error-wrapping only applies to the `RunInference` transform, specifically. > 2. The error wrapping was possible for `RunInference` since most of its work was just calling a single `DoFn`, so it could use the existing `DoFn.with_exception_handling()` method. For transforms that call other transforms, this pattern isn't possible unless every single called transform implements this pattern (which'd require all those transforms to catch their errors this way, etc.) - which basically boils down to forcing every transform to implement its own exception-handling. That doesn't give the option of just adding a top-level error-handler (like my suggestion), which'd be more maintainable IMO. I agree with what you said. I just want to list the current implementations to solve the error handling. And https://github.com/apache/beam/pull/29164 introduces `withBadRecordHandler` for Java to handle IO transforms. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
