Can we think of better name than Transform operator? May be: Expression operator
Reason: We have many operators which are doing 'transform'. So, this name looks very generic. In fact, from the users perspective, I always consider any operator like a black-box doing some transformation. ~ Yogi On 8 March 2016 at 16:46, Mohit Jotwani <[email protected]> wrote: > This can have as many util functions added to it to perform various > transformations to the tuple. > > +1 > > Regards, > Mohit > On 8 Mar 2016 16:42, "Sandeep Deshmukh" <[email protected]> wrote: > > > Looks good to me. > > > > Regards, > > Sandeep > > > > On Mon, Mar 7, 2016 at 9:51 PM, Chinmay Kolhatkar <[email protected]> > > wrote: > > > > > Dear Community, > > > > > > We're creating a transform operator which will allow the apex users to > > > transform data over a stream using this operator. > > > > > > Use Case: > > > ---- > > > When the data has been received from input source, one can transform > data > > > using transform operator and then dump it to destination. > > > > > > Transform means: > > > ---- > > > 1. Conversion of fields from one type to another. Eg. int to Integer, > > epoc > > > to java.util.Date object, int to String or String to int etc. > > > 2. Deriving new fields from one or more input fields. > > > For eg. Input contains 2 fields viz. "firstname", "lastname" and > > > derived field is "fullname" which is combination of firstname and > > lastname > > > in some way. > > > Another example is date of birth present in input tuple, generate > age > > > from it. > > > 3. Change in schema from input to output. For eg. Input schema is > subset > > of > > > output schema and output schema contains some extra derived fields from > > > input fields. > > > > > > > > > Functionality: > > > ---- > > > 1. Transform operator will access POJO as input tuple and emit > > another/same > > > POJO output tuple. > > > 2. Operator needs to be configured with input tuple schema and output > > tuple > > > schema for this. This can be done via TUPLE_CLASS attribute on ports. > > > 3. Generation of output tuple from input tuple can be specified using > > > simple java based expressions. > > > For eg. > > > outfield1 = <expression using fields of input POJO> > > > outfield2 = <expression using fields of output POJO> > > > 4. If no expression is mentioned for a certain output field, then a > > > matching field (name and type) will be picked input POJO and contents > > will > > > be copied to output POJO. > > > Otherwise, the output field will be left to default empty value. > > > > > > > > > Expression Support in PojoUtils: > > > ---- > > > 1. For achieving above functionality, I thought about extending > PojoUtils > > > which already support much of what is required. Then this utility can > be > > > used in Transform operator. > > > 2. Couple of minor enhancements/fixing issues required to this utility: > > > a. Multiple POJO support is not required. Hence will stick to > single > > > POJO. > > > b. PojoUtils does not work well with resolution of getter methods > > when > > > more than one fields are mentioned in expression. Will need to fix > that. > > > c. Following interface can be added to PojoUtils. This will be the > > > return method that can be called for executing the expression > > > interface Expression<O> > > > { > > > O execute(Object obj); > > > } > > > d. Following methods can be added to PojoUtils for addressing > > > expression support: > > > // Evaluates given expression > > > Expression evaluateExpression(String expr, Class > inputObjectType, > > > Class returnType); > > > > > > // Adds a custom method that can be used in an expression. > > > // The one who's using this library can utility this method to > > > provide predefined functionality to user. > > > void registerCustomMethod(String qualifiedFunc) > > > > > > > > > Please share your valuable inputs on this. > > > > > > Thanks, > > > Chinmay. > > > > > >
