Sure Balaji, https://jira.apache.org/jira/browse/HUDI-406 tracks this.
On Fri, Dec 13, 2019 at 4:43 PM Balaji Varadarajan <[email protected]> wrote: > Thanks Shahidha for the quick response. > > Pratyaksh, I am ok with making the behavior consistent with other Key > generators. Please go ahead and submit a PR. > > Thanks, > Balaji.V > > On Thu, Dec 12, 2019 at 10:34 PM Pratyaksh Sharma <[email protected]> > wrote: > > > Hi Shahida, > > > > Thank you for the clarification. Actually I was thinking about a corner > > case where we define the partition field and in some incoming record, the > > value for the corresponding defined partition field is not present. Such > > cases would result in exception and job will get killed. > > > > On Fri, Dec 13, 2019 at 11:02 AM Shahida Khan <[email protected]> > > wrote: > > > > > Hi Pratyaksh, > > > > > > As far as I understand, basic requirement of TimestampBasedKeyGenerator > > is > > > converting the partitions into timebased dateformat. > > > *e.g.* your columns is in Unix Timestamp which need to convert to > > > dateformat like '2019/12/10' > > > > > > There will never be scenario where you won't give partitions and use > > > TimestampBasedKeyGenerator. > > > Also, to use TimestampBasedKeyGenerator, mandate configs needs to be > > define > > > which actually is converting your field to partitions. > > > e.g. > > > hoodie.datasource.write.partitionpath.field= col_dtmDateTime > > > > > > > > > hoodie.datasource.write.keygenerator.class=org.apache.hudi.utilities.keygen.TimestampBasedKeyGenerator > > > hoodie.deltastreamer.keygen.timebased.timestamp.type=EPOCHMILLISECONDS > > > hoodie.deltastreamer.keygen.timebased.output.dateformat=yyyy/MM/dd > > > > > > I hope this help! > > > > > > *Regards,* > > > *Shahida R. Khan* > > > *+91 9167538366* > > > > > > > > > On Thu, 12 Dec 2019 at 12:53, Pratyaksh Sharma <[email protected]> > > > wrote: > > > > > > > Hi, > > > > > > > > If value for configured partitionPathField is not present, we are > > > > defaulting to default partition path in all the key generator classes > > > > except TimestampBasedKeyGenerator. In TimestampBasedKeyGenerator, we > > > > directly throw exception if the value is null. > > > > > > > > I wanted to know if this behaviour is intentional. Ideally we should > > > handle > > > > such cases gracefully everywhere. > > > > > > > > > >
