Balaji Varadarajan created HUDI-1214:
----------------------------------------
Summary: Need ability to set deltastreamer checkpoints when doing
Spark datasource writes
Key: HUDI-1214
URL: https://issues.apache.org/jira/browse/HUDI-1214
Project: Apache Hudi
Issue Type: Improvement
Components: Spark Integration
Reporter: Balaji Varadarajan
Fix For: 0.6.1
Such support is needed for bootstrapping cases when users use spark write to
do initial bootstrap and then subsequently use deltastreamer.
DeltaStreamer manages checkpoints inside hoodie commit files and expects
checkpoints in previously committed metadata. Users are expected to pass
checkpoint or initial checkpoint provider when performing bootstrap through
deltastreamer. Such support is not present when doing bootstrap using Spark
Datasource.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
