Hi Gurudatt,
Good point. There is no specific technical reason behind keeping this class in
hudi-utilities. We can move the core logic to hudi-spark package. Would you be
interested in filing a jira and submitting a PR ?
There are delta-streamer specific configs (TimestampBasedKeyGenerator.Config)
which we need to retain for backwards compatibility reasons but introduce
separate configs - DataSourceWriteOptions (hoodie.datasource.xxx) for
configuring TimestampBasedKeyGenerator as part of datasource write.
Balaji.V
On Wednesday, December 4, 2019, 10:30:40 PM PST, Gurudatt Kulkarni
<[email protected]> wrote:
Hi All,
All other key generators are part of hudi-spark
except TimestampBasedKeyGenerator, it causes issue while using just
hudi-spark directly in a spark job. Any specific reason for this? Can we
move this to hudi-spark module?
Regards,
Gurudatt
