Hi Pratyaksh, Thanks for opening this. Will review and get back to you!
Thanks Vinoth On Sat, Mar 21, 2020 at 2:35 AM Pratyaksh Sharma <[email protected]> wrote: > @Balaji @Vinoth Chandar <[email protected]>, > > Here is a small attempt to make this a generic one - > https://github.com/apache/incubator-hudi/pull/1433/files. Please have a > look, happy to hear from everyone on this. > > This is just a sample, if we agree on the implementation, I will add more > test cases and improve it further. > > On Thu, Feb 27, 2020 at 9:43 PM Vinoth Chandar <[email protected]> wrote: > >> +1 for adding a new composite KeyGenerator, which can combine both... >> >> Workaround : you can use the Transformer api to do a more flexible key >> generation as you wish as well. for deltastreamer >> >> On Tue, Feb 25, 2020 at 9:37 AM Balaji Varadarajan >> <[email protected]> wrote: >> >> > >> > See if you can have a generic implementation where individual fields in >> > the partition-path can be configured with their own key-generator class. >> > Currently, TimestampBasedKeyGenerator is the only type specific custom >> > generator. If we are anticipating more such classes for specialized >> types, >> > you can use a generic way to support overriding key-generator for >> > individual partition-fields once and for all. >> > Balaji.V On Monday, February 24, 2020, 03:09:02 AM PST, Pratyaksh >> > Sharma <[email protected]> wrote: >> > >> > Hi, >> > >> > We have TimestampBasedKeyGenerator for defining custom partition paths >> and >> > we have ComplexKeyGenerator for supporting having combination of fields >> as >> > record key or partition key. >> > >> > However we do not have support for the case where one wants to have >> > combination of fields as record key along with being able to define >> custom >> > partition paths. This use case recently came up at my organisation. >> > >> > How about having CustomTimestampBasedKeyGenerator which supports the >> above >> > use case? This class can simply extend TimestampBasedKeyGenerator and >> allow >> > users to have combination of fields as record key. >> > >> > Open to hearing others' opinions. >> > >> >
