[ https://issues.apache.org/jira/browse/BEAM-681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132142#comment-17132142 ]
Beam JIRA Bot commented on BEAM-681: ------------------------------------ This issue is P2 but has been unassigned without any comment for 60 days so it has been labeled "stale-P2". If this issue is still affecting you, we care! Please comment and remove the label. Otherwise, in 14 days the issue will be moved to P3. Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed explanation of what these priorities mean. > DoFns should be serialized at apply time and deserialized when executing > ------------------------------------------------------------------------ > > Key: BEAM-681 > URL: https://issues.apache.org/jira/browse/BEAM-681 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core > Reporter: Ben Chambers > Priority: P2 > Labels: sdk-consistency, stale-P2 > > 1. Serializing DoFns at application time ensures that any modifications of > fields within the DoFn after application do not accidentally pollute the > execution. This mirrors the approach taken in Java to provide an > approximation of lexical-closure (eg., you only need to know the state of the > DoFn at the time it was applied, not afterwards, to understand its behavior). > 2. Based on 1, the DIrectRunner should also be deserializing DoFns before > running them, which should also detect other classes of errors such as using > the pipeline object (which is not pickleable) within the DoFn -- This message was sent by Atlassian Jira (v8.3.4#803005)