[
https://issues.apache.org/jira/browse/BEAM-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pei He updated BEAM-702:
------------------------
Comment: was deleted
(was: 1. If there are failures in one bundle, we can easily call teardown()s of
DoFns associated with this bundle. (I think we had plan to do it, I am not sure
it has been done or not.)
The tricky part is that if there are failures in other PTransforms or in the
environment while we are processing the bundles. Before runners fail the whole
job, we want runners be able to call teardown() for all running ParDos and
their bundles in processing. We don't have a plan for this part yet.
2. We should document and clarify whether finishBundle() will be called or not
if the bundle fails.)
> Simple pattern for per-bundle and per-DoFn Closeable resources
> --------------------------------------------------------------
>
> Key: BEAM-702
> URL: https://issues.apache.org/jira/browse/BEAM-702
> Project: Beam
> Issue Type: Improvement
> Reporter: Eugene Kirpichov
>
> Dealing with Closeable resources inside a processElement call is easy: simply
> use try-with-resources.
> However, bundle- or DoFn-scoped resources, such as long-lived database
> connections, are less convenient to deal with: you have to open them in
> startBundle and conditionally close in finishBundle (likewise
> setup/teardown), taking special care if there's multiple resources to close
> all of them.
> Perhaps we should provide something like Guava's Closer to DoFn's
> https://github.com/google/guava/wiki/ClosingResourcesExplained. Ideally, the
> user would need to only write a startBundle() or setup() method, but not
> write finishBundle() or teardown() - resources would be closed automatically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
