Thanks everyone for the feedback. @Yogi, the reason for "transform" name is that the tuple will be transformed here.
If there are no further comments on this, I'll start working on this and will create a PR soon. First PR I'll create is for enhancements to PojoUtils as mentioned above and second PR will be for transform operator. Thanks, Chinmay, On Tue, Mar 8, 2016 at 5:48 PM, Devendra Tagare <[email protected]> wrote: > +1 > > Dev > > > On Tue, Mar 8, 2016 at 5:38 PM, Yogi Devendra <[email protected]> > wrote: > > > Forgot to add in earlier email: > > > > +1 for this operator. > > > > ~ Yogi > > > > On 8 March 2016 at 17:03, Yogi Devendra <[email protected]> wrote: > > > > > Can we think of better name than Transform operator? > > > May be: > > > Expression operator > > > > > > Reason: > > > We have many operators which are doing 'transform'. So, this name looks > > > very generic. > > > > > > In fact, from the users perspective, I always consider any operator > like > > a > > > black-box doing some transformation. > > > > > > ~ Yogi > > > > > > On 8 March 2016 at 16:46, Mohit Jotwani <[email protected]> wrote: > > > > > >> This can have as many util functions added to it to perform various > > >> transformations to the tuple. > > >> > > >> +1 > > >> > > >> Regards, > > >> Mohit > > >> On 8 Mar 2016 16:42, "Sandeep Deshmukh" <[email protected]> > > wrote: > > >> > > >> > Looks good to me. > > >> > > > >> > Regards, > > >> > Sandeep > > >> > > > >> > On Mon, Mar 7, 2016 at 9:51 PM, Chinmay Kolhatkar < > [email protected] > > > > > >> > wrote: > > >> > > > >> > > Dear Community, > > >> > > > > >> > > We're creating a transform operator which will allow the apex > users > > to > > >> > > transform data over a stream using this operator. > > >> > > > > >> > > Use Case: > > >> > > ---- > > >> > > When the data has been received from input source, one can > transform > > >> data > > >> > > using transform operator and then dump it to destination. > > >> > > > > >> > > Transform means: > > >> > > ---- > > >> > > 1. Conversion of fields from one type to another. Eg. int to > > Integer, > > >> > epoc > > >> > > to java.util.Date object, int to String or String to int etc. > > >> > > 2. Deriving new fields from one or more input fields. > > >> > > For eg. Input contains 2 fields viz. "firstname", "lastname" > and > > >> > > derived field is "fullname" which is combination of firstname and > > >> > lastname > > >> > > in some way. > > >> > > Another example is date of birth present in input tuple, > > generate > > >> age > > >> > > from it. > > >> > > 3. Change in schema from input to output. For eg. Input schema is > > >> subset > > >> > of > > >> > > output schema and output schema contains some extra derived fields > > >> from > > >> > > input fields. > > >> > > > > >> > > > > >> > > Functionality: > > >> > > ---- > > >> > > 1. Transform operator will access POJO as input tuple and emit > > >> > another/same > > >> > > POJO output tuple. > > >> > > 2. Operator needs to be configured with input tuple schema and > > output > > >> > tuple > > >> > > schema for this. This can be done via TUPLE_CLASS attribute on > > ports. > > >> > > 3. Generation of output tuple from input tuple can be specified > > using > > >> > > simple java based expressions. > > >> > > For eg. > > >> > > outfield1 = <expression using fields of input POJO> > > >> > > outfield2 = <expression using fields of output POJO> > > >> > > 4. If no expression is mentioned for a certain output field, then > a > > >> > > matching field (name and type) will be picked input POJO and > > contents > > >> > will > > >> > > be copied to output POJO. > > >> > > Otherwise, the output field will be left to default empty value. > > >> > > > > >> > > > > >> > > Expression Support in PojoUtils: > > >> > > ---- > > >> > > 1. For achieving above functionality, I thought about extending > > >> PojoUtils > > >> > > which already support much of what is required. Then this utility > > can > > >> be > > >> > > used in Transform operator. > > >> > > 2. Couple of minor enhancements/fixing issues required to this > > >> utility: > > >> > > a. Multiple POJO support is not required. Hence will stick to > > >> single > > >> > > POJO. > > >> > > b. PojoUtils does not work well with resolution of getter > > methods > > >> > when > > >> > > more than one fields are mentioned in expression. Will need to fix > > >> that. > > >> > > c. Following interface can be added to PojoUtils. This will be > > the > > >> > > return method that can be called for executing the expression > > >> > > interface Expression<O> > > >> > > { > > >> > > O execute(Object obj); > > >> > > } > > >> > > d. Following methods can be added to PojoUtils for addressing > > >> > > expression support: > > >> > > // Evaluates given expression > > >> > > Expression evaluateExpression(String expr, Class > > >> inputObjectType, > > >> > > Class returnType); > > >> > > > > >> > > // Adds a custom method that can be used in an expression. > > >> > > // The one who's using this library can utility this > method > > to > > >> > > provide predefined functionality to user. > > >> > > void registerCustomMethod(String qualifiedFunc) > > >> > > > > >> > > > > >> > > Please share your valuable inputs on this. > > >> > > > > >> > > Thanks, > > >> > > Chinmay. > > >> > > > > >> > > > >> > > > > > > > > >
