fhan688 opened a new pull request, #13592: URL: https://github.com/apache/hudi/pull/13592
### Change Logs At present, hudi implemented commit-based rate limiting. In PRD, there are cases where a single commit contains hundreds of thousands of input splits across multiple partitions. Even if commit-level rate limiting is enabled (e.g., read.commits.limit=1), the following issues still exist: - reading all partitions within the commit to build the input split collection is time-consuming. - causing checkpoint timeout This PR adopts split-level rate limiting. ### Impact hudi-flink ### Risk level (write none, low medium or high below) low ### Documentation Update none ### Contributor's checklist -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
