zuston opened a new issue, #901: URL: https://github.com/apache/incubator-uniffle/issues/901
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug When the shuffle server of uniffle colocates with the Yarn nodemanager and using MEMORY_LOCALFILE_HDFS, I found the exception occurs, like this: `Unexpected crc value for blockId xxxxxxxxx` . (Finally, I found this is caused by the partial data into file when meeting the no-space left exception) From the corresponding shuffle-server, I found the disk is full. But why the disk full caused the unexpected crc? I give two possible reasons 1. The flush event is triggered to write the local disk due to the failure of `storage.canWrite()` 2. And then the partial data of one event is written to the file of this disk. The another part data for this event has not been written due to no-space left exception. This makes in-consistency I want to explain more about `stoarge.canWrite()` . Actually, this part logic is correct for exclusive resource deployment for uniffle shuffle-server. But when colocating with other service, this method is incorrect. Because the other service may occupy lots of disk size, which will make the disk limit invalid. ### Affects Version(s) master ### Uniffle Server Log Output _No response_ ### Uniffle Engine Log Output _No response_ ### Uniffle Server Configurations _No response_ ### Uniffle Engine Configurations _No response_ ### Additional context _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
