Re: New testSideInputsWithMultipleWindows and should DoFnRunner explode if DoFn contains a side input ?

2016-12-14 Thread Kenneth Knowles
Yes, this is a bug in SimplerDoFnRunner (or maybe some clarity on whether or not it owns this) not the Spark runner. FWIW the test is definitely correct, and runners-core has had this bug for a while. It is https://issues.apache.org/jira/browse/BEAM-1149 and I'm on it. On Wed, Dec 14, 2016 at 11:0

Re: New testSideInputsWithMultipleWindows and should DoFnRunner explode if DoFn contains a side input ?

2016-12-14 Thread Lukasz Cwik
Yes, it should explode if it has side inputs as well. On Wed, Dec 14, 2016 at 11:03 AM, Amit Sela wrote: > Hi all, > > Yesterday a new test was added to ParDoTest suite: > "testSideInputsWithMultipleWindows". > To the best of my understanding, it's meant to test sideInputs for elements > in mult

New testSideInputsWithMultipleWindows and should DoFnRunner explode if DoFn contains a side input ?

2016-12-14 Thread Amit Sela
Hi all, Yesterday a new test was added to ParDoTest suite: "testSideInputsWithMultipleWindows". To the best of my understanding, it's meant to test sideInputs for elements in multiple windows (unexploded). The Spark runner uses the DoFnRunner (Simple) to process DoFns, and it will explode compres