AlexStocks commented on code in PR #1616:
URL: https://github.com/apache/dubbo-admin/pull/1616#discussion_r4501921808
##########
ai/component/hooks/tracing.go:
##########
@@ -22,15 +22,38 @@ import (
"encoding/json"
"errors"
"fmt"
+ "runtime"
"strconv"
"strings"
+ "sync"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/trace"
)
-const truncatedContentLimit = 4096
+const (
+ // truncatedContentLimit is the size a captured content attribute may
+ // reach. A longer document is shortened before it is attached to a
span.
+ truncatedContentLimit = 4096
+ // truncatedJSONStringLimit is the size of a single string inside a
+ // captured content document.
+ truncatedJSONStringLimit = 512
+)
+
+// AgentName is the human-readable name of the Dubbo Admin SRE agent. The GenAI
+// semantic conventions ask for gen_ai.agent.name on the agent-scoped spans
+// (invoke_agent and execute_tool) and shape the invoke_agent span name as
+// "invoke_agent {gen_ai.agent.name}".
+const AgentName = "dubbo-admin-sre-agent"
+
+// genAIExceptionEvent is the GenAI semantic convention event name for an
+// exception raised during a generative AI client operation.
+const genAIExceptionEvent = "gen_ai.client.operation.exception"
Review Comment:
🟡 **这个事件名用在了规范明确排除的操作上**
`gen-ai-exceptions.md` 对 `gen_ai.client.operation.exception` 的定义是排他的:
> This event represents an exception that occurred during a Generative AI
client operation **other than inference, embeddings, agent invocation, or
response fetching**.
> For inference operations, instrumentations SHOULD record
`gen_ai.client.inference.exception` **instead of this event**.
而 `genAIOperation()` 恰好只覆盖两种操作,两种都该用别的名字:
- `chat <model>`(inference)→
`gen_ai.client.inference.exception`(`client-inference.md:1365`,`The event name
MUST be ...`)
- `execute_tool <tool>` →
`gen_ai.execute_tool.exception`(`execute-tool.md:857`,同样 MUST)
PR 描述里「`gen-ai-exceptions.md`:事件名 MUST
为此」读的是「**这个**事件叫什么」这条约束,不是「所有异常都用它」。按规范建的采集端(把 inference / tool
异常单独成视图的那类)会在这里找不到数据,正好抵消了本 PR「采集端按规范自己认」的核心诉求。
建议把 `genAIOperation` 改成返回事件名:
```go
func genAIExceptionName(event Event) string {
switch event {
case EventModelCallError, EventModelCallEnd: return
"gen_ai.client.inference.exception"
case EventToolCallError, EventToolCallEnd: return
"gen_ai.execute_tool.exception"
}
return ""
}
```
`exceptionRecorder` 的去重不用动。#1618 里 `analyzer.go` 的 `genAIExceptionEvent`
常量要同步支持多个名字,否则那边的错误分类读不到新名字。
##########
ai/component/hooks/tracing.go:
##########
@@ -73,21 +97,129 @@ func newTracingHook(tracer trace.Tracer, captureContent
string) Hook {
span.SetAttributes(stateAttributes(state)...)
if span.IsRecording() {
span.SetAttributes(contentAttributes(state,
captureContent, false)...)
+ // The agent span is closed by the interaction end,
which carries
+ // both the conversation and the final answer. Backends
that read
+ // the trace root (Langfuse does) take the trace input
and output
+ // from this span, so the input has to be emitted here
too.
+ span.SetAttributes(contentAttributes(state,
captureContent, true)...)
}
if state.Error != "" {
span.RecordError(errors.New(state.Error))
span.SetStatus(codes.Error, state.Error)
+ recordGenAIException(span, state, exceptions)
}
switch state.Event {
case EventInteractionEnd, EventIterationEnd, EventStageEnd,
EventModelCallEnd, EventToolCallEnd:
span.End()
+ exceptions.release(span)
default:
span.AddEvent(string(state.Event))
}
return ctx
}
}
+// exceptionRecorder tracks the spans that already carry a GenAI exception
+// event. The agent emits a dedicated error event and the matching end event
+// for a failing operation, and both carry the error, so the recorder keeps the
+// exception reported exactly once per failing operation.
+type exceptionRecorder struct {
+ mu sync.Mutex
+ spans map[spanKey]struct{}
+}
+
+// spanKey identifies a span across traces so recycled span identifiers never
+// collide.
+type spanKey struct {
+ traceID trace.TraceID
+ spanID trace.SpanID
+}
+
+func keyOf(span trace.Span) spanKey {
+ spanContext := span.SpanContext()
+ return spanKey{traceID: spanContext.TraceID(), spanID:
spanContext.SpanID()}
+}
+
+// claim reports whether the caller owns recording the exception event for the
+// span. Only the first caller for a span is granted ownership.
+func (r *exceptionRecorder) claim(span trace.Span) bool {
+ key := keyOf(span)
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ if r.spans == nil {
+ r.spans = make(map[spanKey]struct{})
+ }
+ if _, claimed := r.spans[key]; claimed {
+ return false
+ }
+ r.spans[key] = struct{}{}
+ return true
+}
+
+// release forgets the span once it ended so the recorder does not grow.
+func (r *exceptionRecorder) release(span trace.Span) {
+ key := keyOf(span)
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ delete(r.spans, key)
+}
+
+// recordGenAIException records the GenAI semantic convention exception event
+// on a failing model call or tool call span.
+//
+// The severity the conventions ask for (WARN, severity number 13) cannot be
+// expressed on a span event: the OpenTelemetry Go SDK models an event without
+// a severity field (sdk/trace.Event only carries Name, Attributes and Time)
+// and trace.EventOption only accepts attributes and a timestamp. The span
+// status is therefore left at Error, which is the fallback the GenAI
+// "Recording Errors" document prescribes.
+func recordGenAIException(span trace.Span, state State, exceptions
*exceptionRecorder) {
+ if !genAIOperation(state.Event) || !span.IsRecording() {
+ return
+ }
+ if !exceptions.claim(span) {
+ return
+ }
+ attrs := []attribute.KeyValue{
+ attribute.String("exception.type", exceptionType(state)),
+ attribute.String("exception.message", state.Error),
+ }
+ // The hook only observes the error message, so the stack is captured at
+ // the hook boundary, the same convention the OpenTelemetry Go SDK uses
+ // when it records an exception.
+ if stack := captureStackTrace(); stack != "" {
+ attrs = append(attrs, attribute.String("exception.stacktrace",
stack))
+ }
+ span.AddEvent(genAIExceptionEvent, trace.WithAttributes(attrs...))
+}
+
+// genAIOperation reports whether the event belongs to a generative AI
+// operation that reports client-operation exceptions: a model call or a tool
+// call.
+func genAIOperation(event Event) bool {
+ switch event {
+ case EventModelCallError, EventModelCallEnd, EventToolCallError,
EventToolCallEnd:
+ return true
+ default:
+ return false
+ }
+}
+
+// exceptionType returns the low-cardinality class of the failure. The error
+// type reported by the agent wins; the well-known fallback keeps the
+// attribute non-empty otherwise.
+func exceptionType(state State) string {
+ if state.ErrorType != "" {
+ return state.ErrorType
+ }
+ return errorTypeOther
+}
+
+func captureStackTrace() string {
Review Comment:
🟡 **取的是 hook 自己的栈,不是出错点的栈**
这里在 hook 分发内部调用,所以每一条 `exception.stacktrace` 都长这样:`captureStackTrace →
recordGenAIException → newTracingHook.func1 → Manager.Emit →
...`,跟真正失败的地方(模型超时、工具 panic)没有关系。注释里「the same convention the OpenTelemetry Go
SDK uses」也不准确:SDK 的 `RecordError` 由错误现场调用方触发,栈自然指向现场;在 hook 边界补采是另一种语义。
另外 `runtime.Stack(buffer, false)` 配 2048 字节:buffer 不够时会写入已生成的部分并截断最后一个
frame,结果是「看起来完整、最后一行是半截」,比不写更难排查。
建议二选一:
1. 删掉这个属性(规范里 `exception.stacktrace` 是 `Recommended`,缺失不违规);
2. 在错误产生处带上栈(`errors.WithStack` 或自定义 error),hook 只从 error 里取;要自己采就用
`debug.Stack()`(按需扩容,不会截断)。
按当前实现,每个错误 span 会稳定多出约 2 KB 信息量为零的负载。
##########
ai/component/hooks/manager.go:
##########
@@ -171,6 +183,28 @@ func (m *Manager) HasHooks() bool {
return m != nil && m.hasHooks.Load()
}
+// ReasoningPolicy reports the reasoning capture policy declared by the
+// registered hooks. The most permissive policy wins because every hook that
+// captures payloads shares the same content, and CaptureNone is reported when
+// no hook declares another policy.
+func (m *Manager) ReasoningPolicy() string {
Review Comment:
🟡 **没有生产消费者,注释与实现相互矛盾**
`grep -rn "ReasoningPolicy" --include=*.go ai/ | grep -v _test.go`
除定义处外无命中——只有 `TestManagerAppliesCaptureReasoningPolicy` 和
`TestComponentRegistersCaptureReasoningPolicy` 在用。
更值得改的是注释:
> The most permissive policy wins because **every hook that captures
payloads shares the same content**
而 `Emit` 是**逐 registration** 应用策略的(`ApplyReasoningPolicy(...,
reg.captureReasoning)` / `hookState.reasoningPolicy = reg.captureReasoning`),两个
hook 声明不同策略时看到的内容并不相同。所以这个方法一旦被接上去,返回的是「比任何一个 hook 实际看到的都更宽松」的值,会被下一个 PR 误用。
建议要么删(现在零成本),要么把注释改成「本 manager 对外承诺的最宽策略,不等于单个 hook 的视图」。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]