+1 for approach 2. Regards, Sandeep
On Mon, Mar 28, 2016 at 10:31 PM, Chinmay Kolhatkar <[email protected]> wrote: > +1 for approach 2. > > --- > Sent from mobile. > On 23 Mar 2016 4:25 p.m., "Tushar Gosavi" <[email protected]> wrote: > > > Hi All, > > > > I am planning provide support for adding modules into the json and > property > > file specification of DAG. We will go with the same syntax as discussed > in > > following mail thread. > > > > > > > https://mail-archives.apache.org/mod_mbox/incubator-apex-dev/201512.mbox/%3C565D0E2C.2000805%40datatorrent.com%3E > > > > > > There are multiple choices for the design and I want some suggestion from > > the community aboutg how to go about adding this support. To support > > json/property format, Module should support following functionality > > - Property format uses setSource, addSink methods of StreamMeta, which > were > > not properly supported by Module. > > - Module need capability of extracting port object given names. Operator > > provides this functionality through Operators.describe. > > > > Approach 1) > > As module meta and operator meta shares common fields such as name, port > > information. We can separate this out in a common class NodeMeta and > derive > > OperatorMeta and ModuleMeta from it. This class will handle extracting > port > > information form the object. The changes involve are > > - Split OperatorMeta object > > - PortMeta objects will contain NodeMeta references than > OperatorMeta, > > in some places we will have to perform unchecked cast from NodeMeta to > > OperatorMeta. where code was expecting OperatorMeta from the > > PortMeta. > > - Support setSource and addSink methods. > > - Change Operators.describe to accept Module or Operator and return > > port mapping. > > - Need instanceof call at few places to check if object is Module or > > Operator, before calling specific API while working with property and > json > > file. > > - change signature of few method which accepts Operator to Object. as > > Module and Operator do not share a common parent but requires > > common processing > > - Replace OperatorMeta with NodeMeta in most of the classes. > > > > > > Approach 2) > > Make Module extends Operator and make ModuleMeta extends OperatorMeta. > The > > changes involved will be > > - Change Module interface to extend Operator > > - add support for setSource and addSink for Modules. > > - addOperator will inspect type of object and call addModule if operator > > is a module. > > > > Disadvantage > > - This will break compatibility but this should not be a problem as > Module > > is still an evolving API. > > - Operator lifecyle methods will not get executed for module and we can > > document this. > > > > Advantages > > - This will avoid code duplication at multiple places where there is > > similarity between Module and Operator, and there is lot of similarity > > while defining logical DAG. > > - This will also automatically add support for Module in api being > > developed for operator. For example high level api. > > - This will make module and operator interchangeable in application. > > > > I will prefer Approach 2. > > > > Regards, > > -Tushar. > > >
