[
https://issues.apache.org/jira/browse/HUDI-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677191#comment-17677191
]
Purushotham Pushpavanthar commented on HUDI-2511:
-------------------------------------------------
[~shivnarayan] this issue is still appearing in Hudi 0.11.1. I'm using
`hoodie.cleaner.policy=KEEP_LATEST_BY_HOURS`. Though, we have validation checks
here `org.apache.hudi.config.HoodieWriteConfig.Builder#validate()` but it only
takes care of validating `hoodie.cleaner.commits.retained` against `min and max
commits`. We might need extra checks for the above policy.
> Aggressive archival configs compared to cleaner configs make cleaning moot
> --------------------------------------------------------------------------
>
> Key: HUDI-2511
> URL: https://issues.apache.org/jira/browse/HUDI-2511
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Blocker
> Labels: core-flow-ds, sev:high, user-support-issues
>
> if hoodie.keep.max.commits <= hoodie.cleaner.commits.retained, then cleaner
> will never kick in only. Bcoz, by then archival will kick in and will move
> entries from active to archived.
> We need to revisit this and either throw exception or make cleaner also look
> into archived commits.
> Related issue: [https://github.com/apache/hudi/issues/3739]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
