github-actions[bot] commented on code in PR #66089:
URL: https://github.com/apache/doris/pull/66089#discussion_r3667832941
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/kafka/KafkaUtil.java:
##########
@@ -235,40 +243,14 @@ private static InternalService.PProxyResult
getInfoRequest(InternalService.PProx
try {
while (retryTimes < 3) {
- List<Long> backendIds = new ArrayList<>();
- for (Long beId :
Env.getCurrentSystemInfo().getAllBackendIds(true)) {
- Backend backend =
Env.getCurrentSystemInfo().getBackend(beId);
- if (isBackendAvailableForMetaRequest(backend)
- && !failedBeIds.contains(beId)
- &&
!Env.getCurrentEnv().getRoutineLoadManager().isInBlacklist(beId)) {
- backendIds.add(beId);
- }
- }
- // If there are no available backends, utilize the blacklist.
- // Special scenarios include:
- // 1. A specific job that connects to Kafka may time out for
topic config or network error,
- // leaving only one backend operational.
- // 2. If that sole backend is decommissioned, the
aliveBackends list becomes empty.
- // Hence, in such cases, it's essential to rely on the
blacklist to obtain meta information.
- if (backendIds.isEmpty()) {
- Map<Long, Long> blacklist =
Env.getCurrentEnv().getRoutineLoadManager().getBlacklist();
- for (Long beId : blacklist.keySet()) {
- Backend backend =
Env.getCurrentSystemInfo().getBackend(beId);
- if (isBackendAvailableForMetaRequest(backend) &&
!failedBeIds.contains(beId)) {
- backendIds.add(beId);
- } else if (backend == null) {
- blacklist.remove(beId);
- LOG.warn("remove stale backend {} from routine
load blacklist when getting kafka meta",
- beId);
- }
- }
- }
+ List<Long> candidateBackendIds =
getBackendIdsForMetaRequest(computeGroupName);
Review Comment:
[P2] Count empty-group failures in the metadata metric
`getBackendIdsForMetaRequest` can now throw for an empty cloud compute group
before either `COUNTER_ROUTINE_LOAD_GET_META_FAIL_COUNT` increment, while this
method's `finally` still increments the total-count and latency metrics. A
legacy/replayed job whose saved group cannot be restored therefore fails its
metadata refresh without contributing to the documented failure count,
understating the failure rate. Please account for selection/validation
exceptions in the same exactly-once failure path as no-candidate and
exhausted-retry failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
