dodjdnh opened a new issue, #3774:
URL: https://github.com/apache/dubbo-go/issues/3774

   ## Motivation / Problem
   
   dubbo-go already supports total RPC requests, total failures, and some 
granular error metrics, but it cannot yet independently observe network errors 
(Network) and codec errors (Codec / Serialization). When these errors enter the 
existing RPC metrics collection pipeline, failures whose sources cannot be 
reliably identified fall into Unknown, making it difficult for operators to 
distinguish transport failures from serialization failures.
   
   This issue aims to complete the pipeline from source identification and 
classification to metric export for these two error categories, so that 
Prometheus / Grafana and Dubbo Admin can consume reliable error-classification 
metrics. The goal is not to copy Java's internal implementation, but to add 
verifiable capabilities while preserving compatibility with dubbo-go's existing 
metrics.
   
   ## Current State
   
   The following findings are based on a review of the `develop` source code in 
`apache/dubbo-go` on 2026-10-21.
   
   | Area | Current behavior |
   | --- | --- |
   | Implemented granular error metrics | Timeout, Limit, ServiceUnavailable, 
BusinessFailed, and Unknown, with corresponding windowed statistics |
   | Network / Codec classification | `ErrorTypeNetworkFailure` and 
`ErrorTypeCodec` are reserved, but `classifyError` does not return either 
category |
   | Network / Codec metric definitions | Neither the provider nor consumer 
metric set defines corresponding dedicated cumulative or windowed metrics |
   | Classification counters | The collector has no update branches for these 
two categories |
   | Protocol coverage | Classification currently relies primarily on Triple / 
gRPC status codes; Dubbo protocol classification remains a TODO |
   | Error source identification | `CodeInternal` may originate from transport, 
codec, or other internal errors; some codec errors also use 
`CodeInvalidArgument`, so Network and Codec cannot be distinguished directly by 
status code |
   
   Main source files:
   
   - [Error 
classifier](https://github.com/apache/dubbo-go/blob/develop/metrics/rpc/error_classifier.go)
   - [RPC metric 
definitions](https://github.com/apache/dubbo-go/blob/develop/metrics/rpc/metric_set.go)
   - [RPC metrics 
collection](https://github.com/apache/dubbo-go/blob/develop/metrics/rpc/collector.go)
   
   ## Desired Behavior
   
   1. When the runtime provides reliable structured error information, network 
failures and codec failures can be classified as Network and Codec, 
respectively.
   2. Errors whose sources cannot be reliably determined continue to fall into 
Unknown; do not classify all `CodeInternal` errors as Network or Codec, or 
infer their sources from error-message strings.
   3. Providers and consumers export corresponding cumulative error counts and 
sliding-window error counts for the failure paths they can each observe, with 
explicit names, types, units, labels, and initialization behavior.
   4. Preserve the original errors and their wrapping relationships; monitoring 
classification must not change RPC return statuses, retry policies, or business 
behavior.
   5. Preserve the existing accounting semantics for total failures and 
supported error categories; adding categories must not increment total failures 
more than once.
   
   ## Scope
   
   ### Included
   
   - Inventory where Network / Codec errors originate in the Triple and Dubbo 
protocols, how they propagate, and their visibility to metrics collection.
   - Add reliable error source identification and classification for paths 
confirmed to be coverable, including wrapped errors; safely fall back to 
Unknown when identification is not possible.
   - Complete the corresponding metric definitions, cumulative counter updates, 
and windowed counter updates for providers and consumers.
   - Define the boundaries and precedence between Network and existing 
categories such as Timeout, ServiceUnavailable, and Unknown, avoiding duplicate 
accounting of the same failure in mutually exclusive categories.
   - Add classification unit tests, collector / exporter tests, and validation 
of real RPC failure paths.
   - Document the metric contract, protocol and caller/provider-side coverage, 
enablement, query examples, and compatibility impact.
   
   ### Not Included
   
   - Taking over all Metrics standardization work, renaming existing metrics, 
or making broad changes to existing labels.
   - Implementing all Dubbo protocol error classifications; only the Network / 
Codec paths required by this issue are included, with other categories 
coordinated separately.
   - Extending RPC error metrics into network interface, TCP, or 
operating-system network metrics.
   - Promising complete coverage of all low-level errors that cannot reach RPC 
metrics collection points; limitations and follow-up tasks for these paths 
should be explicitly recorded.
   - Simultaneously modifying Trace / Log consumers, Dubbo Admin product code, 
or Grafana Dashboards in this iteration;
   
   ## Acceptance Criteria
   
   - [ ] Provide an error-path coverage matrix for Triple / Dubbo × provider / 
consumer, identifying which paths can be classified, which cannot reach 
collection points, and the reasons for these limitations.
   - [ ] Establish reproducible error-identification rules for confirmed 
supported Network / Codec paths; do not substitute status codes or error text 
for reliable source information.
   - [ ] Define the contract for new metrics, including names, Counter / Gauge 
types, count units, labels, window semantics, and zero-value / absent-series 
behavior; windowed counts must not be used as cumulative Counters.
   - [ ] Network / Codec failures on paths observable by providers and 
consumers update the corresponding classified metrics, with consistent 
event-accounting semantics between cumulative counts and windowed statistics.
   - [ ] Cover direct errors, wrapped errors, unknown errors, and conflicts 
with categories such as timeout and service unavailable; retain Unknown when 
reliable classification is not possible.
   - [ ] Successful requests do not increment error counters; total failures 
are not incremented more than once; existing Timeout / Limit / 
ServiceUnavailable / BusinessFailed behavior passes regression tests.
   - [ ] New metrics retain bounded RPC identity labels and do not introduce 
high-cardinality labels such as error messages, full URLs, or Trace IDs.
   - [ ] Use real RPC fixtures to trigger controlled network and codec failures 
separately, recording the caller/provider side, protocol, original error, final 
classification, and exporter / Prometheus metric changes; manually constructed 
classifier inputs alone must not replace integration validation.
   - [ ] Record validation failures and uncovered cases separately; do not 
treat a Dashboard's zero-value fallback as proof that the metric capability has 
passed validation.
   - [ ] Relevant unit tests, static checks, and concurrency checks pass, and 
metric usage and compatibility documentation are provided.
   
   These are proposed acceptance criteria; implementation and validation have 
not yet been completed.
   
   ## Context / Design Notes
   
   This issue is an independent subtask within the observability plan. It does 
not replace or close the following overarching issues:
   
   - **Related observability plan: [ #3562 — Complete dubbo-go observability 
implementation and end-to-end 
integration](https://github.com/apache/dubbo-go/issues/3562)**, covering shared 
error semantics and Metrics completion.
   - **Related Metrics task: [ #3337 — Improve Metrics coverage and 
standardization](https://github.com/apache/dubbo-go/issues/3337)**, which 
explicitly proposes distinguishing Network / Codec errors and is currently 
assigned to @Saramanda9988.
   - **Overlap coordination: [ 
#3568](https://github.com/apache/dubbo-go/issues/3568)**, which includes Dubbo 
protocol error classification; duplicate changes to the classifier should be 
avoided.
   
   Existing work should be reused: 
[#3189](https://github.com/apache/dubbo-go/pull/3189) provides the foundation 
for the existing granular error metrics; 
[#3603](https://github.com/apache/dubbo-go/pull/3603) and 
[#3665](https://github.com/apache/dubbo-go/pull/3665) add classification 
semantics documentation and tests, but do not complete the dedicated Network / 
Codec metrics.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to