Thank you folks for the fast response and work. Vinoth Chandar <[email protected]> escreveu no dia quarta, 25/03/2020 à(s) 11:22:
> btw Raymond already has a PR up here for this :) > https://github.com/apache/incubator-hudi/pull/1440 > > On Mon, Mar 23, 2020 at 5:32 PM Shiyan Xu <[email protected]> > wrote: > > > Seems like an abstract class would be good enough for generic use? > > User can provide a list of `Transformer` then the abstract class just > apply > > all the way through the list. > > The implementation can be minimal for this approach. > > > > On Mon, Mar 23, 2020 at 4:12 PM Vinoth Chandar <[email protected]> > wrote: > > > > > sg. Filed https://issues.apache.org/jira/browse/HUDI-731 > > > > > > Someone looking to pick this? :). Its an nice feature to implement, > that > > > fits a good template.. > > > > > > ofc we can discuss this more here in parallel > > > > > > On Mon, Mar 23, 2020 at 8:31 AM FO O <[email protected]> wrote: > > > > > > > Thank you Vinoth. > > > > > > > > >"If you are talking about implementing support for chained calling > of > > > > multiple Transformers, within DeltaStreamer itself" > > > > > > > > Yes, chained calling support for transformers would be super helpful, > > if > > > > this discussion can be revived it would be great. > > > > > > > > I see this useful for folks using DMS transformer and that need some > > kind > > > > of transformation before the DMS transformer adds the op filed for > > > initial > > > > load or when loading the CDC. In the meantime, I will create a custom > > > > transformer. > > > > > > > > Thanks again, > > > > -F. > > > > > > > > > > > > Vinoth Chandar <[email protected]> escreveu no dia domingo, > 22/03/2020 > > > > à(s) > > > > 20:58: > > > > > > > > > Hi F, > > > > > > > > > > The Transformer interface allows you to basically plugin anything > > that > > > > > takes a DataFrame and returns a transformed DataFrame. Does that > > help? > > > > > If you are talking about implementing support for chained calling > of > > > > > multiple Transformers, within DeltaStreamer itself..It has been > > > discussed > > > > > before. > > > > > And we can revive that conversation. > > > > > > > > > > Thanks > > > > > Vinoth > > > > > > > > > > On Sat, Mar 21, 2020 at 5:14 PM FO O <[email protected]> wrote: > > > > > > > > > > > Hi team! > > > > > > > > > > > > My use case would benefit from running a SQL transformer followed > > by > > > > the > > > > > > DMS transformer. > > > > > > > > > > > > It seems my best options is to create a new transformer that is > > based > > > > on > > > > > > the current DMS transformer and add the additional > transformation I > > > > need > > > > > > (add new columns, concatenate fields). > > > > > > > > > > > > Wanted to see if there are additional recommendations that I > should > > > > > > consider instead of this one. > > > > > > > > > > > > Thank you, > > > > > > F > > > > > > > > > > > > > > > > > > > > >
