This is terrific! Thanks for the proposal. On Tue, Jun 28, 2016 at 9:06 AM, Thomas Groh <[email protected]> wrote:
> Hey Everyone: > > We've recently started to be permitted to reuse DoFn instances in Beam[1]. > Beyond the efficiency gains from not having to deserialize new DoFn > instances for every bundle, DoFn reuse also provides the ability to > minimize expensive setup work done per-bundle, which hasn't formerly been > possible. Additionally, it has also enabled more failure cases, where > element-based state leaks improperly across bundles. > > I've written a document proposing that two methods are added to the API of > DoFn, setup and teardown, which both provides hooks for users to write > efficient DoFns, as well as signals that DoFns will be reused. > > The document is located at > > https://docs.google.com/document/d/1LLQqggSePURt3XavKBGV7SZJYQ4NW8yCu63lBchzMRk/edit?ts=5771458f# > and committers have edit access > > Thanks, > > Thomas > > [1] https://github.com/apache/incubator-beam/pull/419 >
