waitingF opened a new pull request, #10459: URL: https://github.com/apache/hudi/pull/10459
### Change Logs When writing data with BoundedInMemoryQueue, As I tested, about 9% CPU is used to estimaze the object size to calculate the queue size. Currently the sampling rate is set to 64. If we can set the rate to 128, then we can save 4.5% CPU theoretically.  ### Impact improve performance by setting higher sampling rate ### Risk level (write none, low medium or high below) low ### Documentation Update - Add a new configuration named `hoodie.write.buffer.record.sampling.rate`, default 64. Higher value leads to lower CPU usage for object size estimation. - Add a new config named `hoodie.write.buffer.record.cache.limit` for better memory footprint control, default 128*1024. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
