[
https://issues.apache.org/jira/browse/HUDI-8830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jack Lei updated HUDI-8830:
---------------------------
Description: currently, we use now() - splitLatestCommit, however, when the
time goes and the task just processes a huge data commit, then the diff between
now and splitLatestCommit may get larger. so we may let reader operator holds a
maxLatestCommitTime attribute to record the max split latestcommit time has
got, and then use it to (was: currently, we use now() - splitLatestCommit,
however, when the time goes and the task just processes a huge data commit,
then the diff between now and splitLatestCommit may get larger. so we may just
use the max)
> use maxLatestCommitTime to compute splitLatestCommitDelay
> ---------------------------------------------------------
>
> Key: HUDI-8830
> URL: https://issues.apache.org/jira/browse/HUDI-8830
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: jack Lei
> Priority: Major
>
> currently, we use now() - splitLatestCommit, however, when the time goes and
> the task just processes a huge data commit, then the diff between now and
> splitLatestCommit may get larger. so we may let reader operator holds a
> maxLatestCommitTime attribute to record the max split latestcommit time has
> got, and then use it to
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
