Hi Shahida, Thank you for the clarification. Actually I was thinking about a corner case where we define the partition field and in some incoming record, the value for the corresponding defined partition field is not present. Such cases would result in exception and job will get killed.
On Fri, Dec 13, 2019 at 11:02 AM Shahida Khan <[email protected]> wrote: > Hi Pratyaksh, > > As far as I understand, basic requirement of TimestampBasedKeyGenerator is > converting the partitions into timebased dateformat. > *e.g.* your columns is in Unix Timestamp which need to convert to > dateformat like '2019/12/10' > > There will never be scenario where you won't give partitions and use > TimestampBasedKeyGenerator. > Also, to use TimestampBasedKeyGenerator, mandate configs needs to be define > which actually is converting your field to partitions. > e.g. > hoodie.datasource.write.partitionpath.field= col_dtmDateTime > > hoodie.datasource.write.keygenerator.class=org.apache.hudi.utilities.keygen.TimestampBasedKeyGenerator > hoodie.deltastreamer.keygen.timebased.timestamp.type=EPOCHMILLISECONDS > hoodie.deltastreamer.keygen.timebased.output.dateformat=yyyy/MM/dd > > I hope this help! > > *Regards,* > *Shahida R. Khan* > *+91 9167538366* > > > On Thu, 12 Dec 2019 at 12:53, Pratyaksh Sharma <[email protected]> > wrote: > > > Hi, > > > > If value for configured partitionPathField is not present, we are > > defaulting to default partition path in all the key generator classes > > except TimestampBasedKeyGenerator. In TimestampBasedKeyGenerator, we > > directly throw exception if the value is null. > > > > I wanted to know if this behaviour is intentional. Ideally we should > handle > > such cases gracefully everywhere. > > >
