afterincomparableyum opened a new pull request, #3583:
URL: https://github.com/apache/celeborn/pull/3583
<!--
Thanks for sending a pull request! Here are some tips for you:
- Make sure the PR title start w/ a JIRA ticket, e.g. '[CELEBORN-XXXX]
Your PR title ...'.
- Be sure to keep the PR description updated to reflect all changes.
- Please write your PR title to summarize what this PR proposes.
- If possible, provide a concise example to reproduce the issue for a
faster review.
-->
This PR implements retry support for createReader failures in the C++
client, matching the behavior of the Java implementation. The implementation
includes:
- Added configuration properties:
* clientFetchMaxRetriesForEachReplica (default: 3)
* dataIoRetryWait (default: 5s)
* clientPushReplicateEnabled (default: false)
- Added peer location support methods to PartitionLocation:
* hasPeer() - Check if location has a peer replica
* getPeer() - Get the peer location
* hostAndFetchPort() - Get host:port string for logging
- Implemented retry logic in createReaderWithRetry():
* Retries up to fetchChunkMaxRetry_ times (doubled if replication
enabled)[which is why I added this parameter in this PR]
* Switches to peer location on failure when available
* Sleeps between retries when both replicas tried or no peer exists
* Resets retry counter when moving to new location or on success
- Added unit tests for new functionality
### How was this patch tested?
Unit tests and compiling
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
