The JIRA is here https://issues.apache.org/jira/browse/APEXMALHAR-2085, which has all the information.
On Fri, May 13, 2016 at 4:06 PM, Thomas Weise <[email protected]> wrote: > Maybe put everything into a google doc and share the link? > > > On Fri, May 13, 2016 at 4:04 PM, David Yan <[email protected]> wrote: > > > Making Siyuan's diagram fixed width so we can actually see it :) > > > > +-------------------+ > > +---------> | WindowedOperator | <--------+ > > | +--------+----------+ | > > | ^ ^--------------------------------+ > > | | | | > > | | | | > > +------+--------+ +------+------+ +-------+-----+ > +------+-----+ > > |CombineOperator| |GroupOperator| |KeyedOperator| > |JoinOperator| > > +---------------+ +-------------+ +------+------+ > +-----+------+ > > +---------^ ^ ^ > > | | | > > +--------+---+ +-----+----+ +----+----+ > > |KeyedCombine| |KeyedGroup| | CoGroup | > > +------------+ +----------+ +---------+ > > > > On Wed, May 11, 2016 at 11:47 AM, Siyuan Hua <[email protected]> > > wrote: > > > > > Hi guys, > > > > > > I just create a ticket for windowed operators. It is the prerequisite > for > > > High-level API and Beam integration. > > > > > > As per our recent several discussions in the community. A group of > > Windowed > > > Operators that delivers the window semantic follows the google Data > Flow > > > model(https://cloud.google.com/dataflow/) is very important. > > > The operators should be designed and implemented in a way for > > > High-level API > > > Beam translation > > > Easy to use with other popular operator > > > > > > Hierarchy of the operators, > > > The windowed operators should cover all possible transformations that > > > require window, and batch processing is also considered as special > window > > > called global window > > > > > > +-------------------+ > > > +---------> | WindowedOperator | <--------+ > > > | +--------+----------+ | > > > | ^ ^--------------------------------+ > > > | | | | > > > | | | | > > > +------+--------+ +------+------+ +-------+-----+ > > +------+-----+ > > > |CombineOperator| |GroupOperator| |KeyedOperator| > > |JoinOperator| > > > +---------------+ +-------------+ +------+------+ > > +-----+------+ > > > +---------^ ^ ^ > > > | | | > > > +--------+---+ +-----+----+ > +----+----+ > > > |KeyedCombine| |KeyedGroup| | > CoGroup | > > > +------------+ +----------+ > +---------+ > > > > > > > > > Combine operation includes all operations that combine all tuples in > one > > > window into one or small number of tuples, Group operation group all > > tuples > > > in one window, Join and CoGroup are used to join and group tuples from > > > different inputs. > > > > > > Components: > > > Window Component, it includes configuration, window state that should > be > > > checkpointed, etc. It should support NonMergibleWindow(fixed or slide) > > > MergibleWindow(Session) > > > > > > Trigger: > > > It should support early trigger, late trigger. It should support > > > customizable trigger behaviour > > > > > > Outside components: > > > Watermark generator, can be plugged into input source to generate > > watermark > > > Tuple util components, The operator should be only working on Tuples > with > > > schema. It can handle either predefined tuple type or give a > declarative > > > API to describe the user defined tuple class > > > > > > Most component API should be reused in High-Level API > > > > > > This is the umbrella ticket, separate tickets would be created for > > > different components and operators respectively > > > > > >
