SteNicholas opened a new pull request, #3525: URL: https://github.com/apache/celeborn/pull/3525
### What changes were proposed in this pull request? `ReadBufferDispatcher` should add timeout constraints to fast fail in case of timeout. ### Why are the changes needed? Setting `celeborn.worker.directMemoryRatioForReadBuffer` with small ratio may result in a backlog of read buffer requests for `ReadBufferDispatcher` at present, which causes running flink jobs to stall. `ReadBufferDispatcher` should add timeout constraints to fast fail in case of timeout, avoiding long wait times for client. ### Does this PR resolve a correctness bug? No. ### Does this PR introduce _any_ user-facing change? Introduce `celeborn.worker.readBuffer.processTimeout` config to specify timeout for buffer dispatcher to process a read buffer request. ### How was this patch tested? `ReadBufferDispactherSuite#[CELEBORN-2192] ReadBufferDispatcher should add timeout constraints to fast fail in case of timeout` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
