gaoyuan5251 opened a new pull request, #3195:
URL: https://github.com/apache/hugegraph/pull/3195
## Purpose of the PR
- fix #3172
In 1.7.0, after a cluster-wide PD restart the raft leader may land on any
node. Followers then fail to redirect requests to the new leader:
`RaftEngine.getLeaderGrpcAddress()` NPEs because the bolt `GetMemberRequest`
RPC response is not null-checked and `future.get()` has no timeout, and
follower PDs keep logging `redirect to leader with error: null`. Additionally,
when the member query bolt RPC fails (even for localhost), `getMembers()` marks
**all** members Offline, so `/v1/members` reports `stateCountMap: {"Offline":
3}` and `pdLeader: null` while the raft cluster itself is healthy.
This PR fixes the two main symptoms described in #3172 on the
`release-1.7.0` branch. Note that master already contains partial null-safety
for `getLeaderGrpcAddress()` via #2961, but `getMembers()` is still unprotected
there, and none of these fixes exist in 1.7.0.
## Main Changes
- **RaftEngine**: build a static mapping (raft endpoint -> gRPC address)
from the local config at init time, so address resolution no longer depends on
the fragile bolt RPC (which can fail even for localhost per #3172)
- Cache the resolved leader gRPC address and invalidate the cache on raft
leader change via listener
- `getMembers()`: prefer the static mapping; keep the bolt RPC as fallback
for peers missing from the mapping
- **ServiceGrpc**: complete the gRPC observer with
`onError(Status.UNAVAILABLE)` when redirect-to-leader fails, so client requests
fail fast instead of hanging forever
## Verifying these changes
- [ ] Trivial rework / code cleanup without any test coverage. (No Need)
- [ ] Already covered by existing tests, such as *(please modify tests
here)*.
- [x] Need tests and can be verified as follows:
- Deployed on a 3-node PD cluster (CentOS 7.8, JDK 11) built from this
branch
- Restart the whole PD cluster so the leader moves to a non-first node:
follower redirect works, no more `redirect to leader with error` NPE
- `GET /v1/members` on the leader returns all members `Up` with
grpcUrl/restUrl filled in and the correct `pdLeader`
## Does this PR potentially affect the following parts?
- [ ] Dependencies
- [ ] Modify configurations
- [ ] The public API
- [x] Other affects (PD internal leader address resolution & follower
redirect path)
- [ ] Nope
## Documentation Status
- [x] `Doc - No Need`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
