vinishjail97 opened a new pull request, #10777: URL: https://github.com/apache/hudi/pull/10777
### Change Logs In our current code we are doing a coalesce which just decreases the partitions but doesn't increase them, adding a function known as `coalesceOrRepartition` which does coalesce or repartition depending on the rdd partitions and the numPartitions calculated using the task/partition size. ### Impact Improvement in S3/GCS sources to increase/decrease parallelism based on partition size. ### Risk level (write none, low medium or high below) Medium ### Documentation Update None. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Change Logs and Impact were stated clearly - [x] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
