[
https://issues.apache.org/jira/browse/HUDI-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason updated HUDI-5906:
---------------------------------
Fix Version/s: 0.14.1
(was: 0.14.0)
> Consolidate checkpointing logic with deltastreamer into CheckpointStrategy
> model
> --------------------------------------------------------------------------------
>
> Key: HUDI-5906
> URL: https://issues.apache.org/jira/browse/HUDI-5906
> Project: Apache Hudi
> Issue Type: Improvement
> Components: deltastreamer
> Reporter: Raymond Xu
> Priority: Major
> Fix For: 0.14.1
>
>
> checkpointing logic with deltastreamer is unorganized. there are logic spread
> in different places
> - HoodieDeltaStreamer#CHECKPOINT_KEY
> - HoodieDeltaStreamer#CHECKPOINT_RESET_KEY
> - HoodieDeltaStreamer#CHECKPOINT_FORCE_SKIP_PROP
> -
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.Config#checkpoint
> -
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.Config#initialCheckpointProvider
> -
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.Config#allowCommitOnNoCheckpointChange
> - MUTLI_WRITER_SOURCE_CHECKPOINT_ID
> - org.apache.hudi.utilities.checkpointing.InitialCheckPointProvider
> -
> org.apache.hudi.utilities.sources.HoodieIncrSource.Config#MISSING_CHECKPOINT_STRATEGY
> -
> org.apache.hudi.utilities.sources.helpers.IncrSourceHelper.MissingCheckpointStrategy
> There should be a consolidated model say HoodieCheckpointStrategy to manage
> all these logic with pluggable APIs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
