Seems like an abstract class would be good enough for generic use? User can provide a list of `Transformer` then the abstract class just apply all the way through the list. The implementation can be minimal for this approach.
On Mon, Mar 23, 2020 at 4:12 PM Vinoth Chandar <[email protected]> wrote: > sg. Filed https://issues.apache.org/jira/browse/HUDI-731 > > Someone looking to pick this? :). Its an nice feature to implement, that > fits a good template.. > > ofc we can discuss this more here in parallel > > On Mon, Mar 23, 2020 at 8:31 AM FO O <[email protected]> wrote: > > > Thank you Vinoth. > > > > >"If you are talking about implementing support for chained calling of > > multiple Transformers, within DeltaStreamer itself" > > > > Yes, chained calling support for transformers would be super helpful, if > > this discussion can be revived it would be great. > > > > I see this useful for folks using DMS transformer and that need some kind > > of transformation before the DMS transformer adds the op filed for > initial > > load or when loading the CDC. In the meantime, I will create a custom > > transformer. > > > > Thanks again, > > -F. > > > > > > Vinoth Chandar <[email protected]> escreveu no dia domingo, 22/03/2020 > > à(s) > > 20:58: > > > > > Hi F, > > > > > > The Transformer interface allows you to basically plugin anything that > > > takes a DataFrame and returns a transformed DataFrame. Does that help? > > > If you are talking about implementing support for chained calling of > > > multiple Transformers, within DeltaStreamer itself..It has been > discussed > > > before. > > > And we can revive that conversation. > > > > > > Thanks > > > Vinoth > > > > > > On Sat, Mar 21, 2020 at 5:14 PM FO O <[email protected]> wrote: > > > > > > > Hi team! > > > > > > > > My use case would benefit from running a SQL transformer followed by > > the > > > > DMS transformer. > > > > > > > > It seems my best options is to create a new transformer that is based > > on > > > > the current DMS transformer and add the additional transformation I > > need > > > > (add new columns, concatenate fields). > > > > > > > > Wanted to see if there are additional recommendations that I should > > > > consider instead of this one. > > > > > > > > Thank you, > > > > F > > > > > > > > > >
