This is a bit of an improvised change to the Beam model, if these are really treated *that* specially. (notably, they are a subset of the WindowFns that we ship with our SDKs, so it really is a careful selection)
It does make sense to have some special WindowFns with distinguished semantics, since a lot of use cases are covered without the generality of the Beam model. Having a compact proto repr for these makes sense, but the particular choice of functions I had viewed as a hack while the Fn API was maturing to support the real model. If this is a new special category in the model (capability matrix?) there should be a lot more input and deliberation as to what they are. TBH I think we will probably choose just the WindowFns that you did, but the standardization deserves attention and documentation. Kenn On Wed, Jul 26, 2017 at 9:34 PM, Robert Bradshaw < [email protected]> wrote: > I think there may be a distinction between hard-coding support for the > "standard" WindowFns (e.g. > https://github.com/apache/beam/blob/master/sdks/common/ > runner-api/src/main/proto/standard_window_fns.proto) > and accepting WindowFns as a UDF. Different runners have offered different > levels of support for this in the past (for example, the Fn API doesn't > support WindowFns other than these standard ones unless they're implemented > in Java--and even then it'd probably be better for these to be executed in > the SDK context). > > On Wed, Jul 26, 2017 at 9:16 PM, Kenneth Knowles <[email protected]> > wrote: > > > Hi Etienne, > > > > Every WindowFn is a UDF, so there is really no such thing as "custom" > > window merging. Is this the same as saying that a runner supports only > > merging for Sessions? Or just supports WindowFn that merges based on > > overlap? > > > > Kenn > > > > On Mon, Jul 24, 2017 at 10:15 AM, Etienne Chauchot <[email protected]> > > wrote: > > > > > Hi all, > > > > > > There is now 2 new ValidatesRunner tests: WindowTest. > > testMergingCustomWindows > > > and WindowTest.testMergingCustomWindowsKeyedCollection. The aim of > these > > > tests is to verify that the runners can handle custom windowFn > > (extensions > > > of windowFn that, for example, could rely on elements in addition to > > > timestamps). > > > > > > As new runners are coming, I wanted to let you know that there is also > a > > > new category tag UsesCustomWindowMerging that you can use to skip these > > > tests while running ValidatesRunner tests on runners that do not > support > > > custom window merging yet. > > > > > > Besides, there is also an ongoing related PR ( > > > https://github.com/apache/beam/pull/3592) to enhance the test utils > > > methods of WindowFnTestUtils. > > > > > > Etienne > > > > > > > > > > > > > > > > > >
