leixm commented on PR #1315:
URL:
https://github.com/apache/incubator-uniffle/pull/1315#issuecomment-1818183961
> > When an app is writing data and restarting ShuffleServer, used_buffer
cannot be released, because a NO_REGISTER exception will be thrown when
sendingShuffleData. At this time, only preAllocated is released, but
used_buffer is not released.
>
> Could you elaborate a bit more? Per my understanding, if the shuffle
server is restarting, the server would fail the writer at L210-L219? How is it
possible to bypass that check and fails at `cacheShuffleData`?
In our production environment, what caused the used_buffer leak was the
re-requireBuffer part. Sorry, I checked the code and found that #1045 can also
fix this problem. ShuffleTaskManager#requireBuffer `if (null ==
shuffleTaskInfo) {
return RequireBufferStatusCode.NO_REGISTER.statusCode();
}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
