[ https://issues.apache.org/jira/browse/APEXMALHAR-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Yan resolved APEXMALHAR-2085. ----------------------------------- Resolution: Fixed Fix Version/s: 3.5.0 > Implement Windowed Operators > ---------------------------- > > Key: APEXMALHAR-2085 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2085 > Project: Apache Apex Malhar > Issue Type: New Feature > Reporter: Siyuan Hua > Assignee: David Yan > Fix For: 3.5.0 > > > As per our recent several discussions in the community. A group of Windowed > Operators that delivers the window semantic follows the google Data Flow > model(https://cloud.google.com/dataflow/) is very important. > The operators should be designed and implemented in a way for > High-level API > Beam translation > Easy to use with other popular operator > {panel:title=Operator Hierarchy} > Hierarchy of the operators, > The windowed operators should cover all possible transformations that require > window, and batch processing is also considered as special window called > global window > {code} > +-------------------+ > +---------> | WindowedOperator | <--------+ > | +--------+----------+ | > | ^ ^--------------------------------+ > | | | | > | | | | > +------+--------+ +------+------+ +-------+-----+ +------+-----+ > |CombineOperator| |GroupOperator| |KeyedOperator| |JoinOperator| > +---------------+ +-------------+ +------+------+ +-----+------+ > +---------^ ^ ^ > | | | > +--------+---+ +-----+----+ +----+----+ > |KeyedCombine| |KeyedGroup| | CoGroup | > +------------+ +----------+ +---------+ > {code} > Combine operation includes all operations that combine all tuples in one > window into one or small number of tuples, Group operation group all tuples > in one window, Join and CoGroup are used to join and group tuples from > different inputs. > {panel} > {panel:title=Components} > * Window Component > It includes configuration, window state that should be checkpointed, etc. It > should support NonMergibleWindow(fixed or slide) MergibleWindow(Session) > * Trigger > It should support early trigger, late trigger with customizable trigger > behaviour > * Other related components: > ** Watermark generator, can be plugged into input source to generate watermark > ** Tuple schema support: > It should handle either predefined tuple type or give a declarative API to > describe the user defined tuple class > {panel} > Most component API should be reused in High-Level API > This is the umbrella ticket, separate tickets would be created for different > components and operators respectively -- This message was sent by Atlassian JIRA (v6.3.4#6332)