wForget opened a new issue, #2763: URL: https://github.com/apache/uniffle/issues/2763
### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [x] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug In the Gluten shuffle writer, [`WriteBufferManager.clear()`](https://github.com/apache/uniffle/blob/018ac289687079defee1c0631bb9c3ad96c0f58c/client-spark/common/src/main/java/org/apache/spark/shuffle/writer/WriteBufferManager.java#L373) is called during the final cleanup. Currently, this method uses the configured `bufferSpillRatio`. As this is the final call, using a ratio lower than `1.0` may leave some buffered data unflushed. The spill ratio should therefore be set to `1.0` in `clear()` to ensure that all remaining buffered data is converted into shuffle blocks and flushed. ### Affects Version(s) master ### Uniffle Server Log Output ```logtalk ``` ### Uniffle Engine Log Output ```logtalk ``` ### Uniffle Server Configurations ```yaml ``` ### Uniffle Engine Configurations ```yaml ``` ### Additional context _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
