Daniel Halperin resolved BEAM-801.
       Resolution: Fixed
    Fix Version/s: Not applicable

> SplittableParDo must be a pseudo-primitive, not ParDo(OldDoFn)
> --------------------------------------------------------------
>                 Key: BEAM-801
>                 URL: https://issues.apache.org/jira/browse/BEAM-801
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-core, runner-direct
>            Reporter: Kenneth Knowles
>            Assignee: Eugene Kirpichov
>             Fix For: Not applicable
> Today, runners-core contains an expansion of {{ParDo(DoFn)}} that implements 
> it via unsupported features of {{ParDo(OldDoFn)}}. This expansion is used by 
> the DirectRunner.
> The right approach to provide a ready implementation in runners-core is the 
> one taken by {{GroupAlsoByWindowViaWindowSetDoFn}} where the unsupported 
> features are provided to the constructor of the {{DoFn}}, rather than 
> expected to be passed through the {{ProcessContext}}.
> These features are not guaranteed to be part of the public state & timers API 
> of {{DoFn}}, particularly in the case of watermark holds, so it is not 
> prudent to plan on waiting for the new state & timers API and porting over.
> These issues create real blockers by causing dependency cycles between 
> implementing {{DoFn}} fully (requires no use of unsupported features), 
> implementing new state and timers (requires new {{DoFn}}), and keeping the 
> hack until new {{DoFn}} has state and timers (requires use of unsupported 
> features).
> To break the cycle, the tests that rely on unsupported features are being 
> disabled. They can be re-enabled either by following the design suggested 
> above (probably the most robust approach) or by waiting and porting to new 
> features as they are available.

This message was sent by Atlassian JIRA

Reply via email to