prashantwason opened a new pull request, #8527: URL: https://github.com/apache/hudi/pull/8527
[HUDI-6117] Parallelize the initial creation of file groups for a new MDT partition. ### Change Logs File group creation is parallelized using engineContext.foreach. Previous leftover files in the MDT partition are deleted before creation. ### Impact Faster file group creation when there are a large number of file groups for a MDT partition. Fixes the issue where previous failed initialization could have left over partially or wholly written log files with different instant time. ### Risk level (write none, low medium or high below) None ### Documentation Update None ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
