jerqi opened a new issue, #907: URL: https://github.com/apache/incubator-uniffle/issues/907
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues. ### What would you like to be improved? We already have `PooledHadoopShuffleWriteHandler`, if the number of flushEvent reach the maxConcurrency, the thread will be blocked now. Every handler will close the file after it write one flushEvent. If we have many flushEvents, we don't need to close file every time. We can increase a queue, if the number of flushEvent reach the maxConcurrency, we add the event to the queue, the handler finish writing one flushEvent, we check whether the queue is empty. If the queue isn't empty, we continue taking one event to write. If the queue is empty, we close the file. ### How should we improve? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
