Croway opened a new pull request, #30291:
URL: https://github.com/apache/camel/pull/30291

   [CAMEL-27131](https://issues.apache.org/jira/browse/CAMEL-27131), sub-task 
of [CAMEL-23369](https://issues.apache.org/jira/browse/CAMEL-23369).
   
   Today the AI producers record one span per LLM call with the model, the 
token usage and the finish reason. That stays the default. This PR adds an 
opt-in level for tracing backends built around LLM traces (Langfuse, and the 
others that read the OpenTelemetry GenAI semantic conventions), and stops 
recording twice the calls that an AI framework already traces. Before/after 
screenshots from Langfuse are in [the JIRA 
comment](https://issues.apache.org/jira/browse/CAMEL-27131?focusedCommentId=18146297#comment-18146297).
   
   ### Observability levels: `camel.aiObservability.level`
   | Level | Records |
   |---|---|
   | `basic` (default) | as today |
   | `detailed` | an `invoke_agent` span per `langchain4j-agent` run and per 
`openai` tool loop, one `chat` span per model call, an `execute_tool` span per 
tool call (Camel route tools through `AiToolExecutor`, so every component; MCP 
tools of `openai`), request parameters, cached tokens, `gen_ai.provider.name`, 
`gen_ai.conversation.id` |
   | `full` | `gen_ai.input.messages`, `gen_ai.output.messages`, 
`gen_ai.system_instructions`, tool arguments and results, each text cut at 
`camel.aiObservability.contentMaxLength` (8192) |
   
   Metrics are the same at every level: content and ids never become metric 
tags, and agent runs and tool calls are not GenAI client operations (through 
Micrometer they are Observations named `camel.ai.agent` / `camel.ai.tool`).
   
   ### No double recording: `camel.aiObservability.deferToFramework` (default 
`true`)
   Camel no longer records the calls that are already traced by:
   - Spring AI, when the model holds a non no-op `ObservationRegistry` (model 
calls and route tools);
   - quarkus-langchain4j (`SpanChatModelListener`) or `langchain4j-observation` 
(`ObservationChatModelListener`) on the LangChain4j model;
   - the OpenTelemetry Java agent's `openai-java` instrumentation (chat 
completions and embeddings of `camel-openai`), except at level `full`, where 
Camel keeps its span (the agent writes the messages to log events only) and 
warns once.
   
   At `detailed` the agent run and the route tools are still Camel's, and the 
framework's spans nest under the run.
   
   ### At every level
   - token counts are numeric attributes; on the Micrometer Observation path 
the usage, response model and finish reason are high-cardinality values (as 
Spring AI does);
   - when `camel-opentelemetry2` is in use, GenAI spans go through it even if 
an `ObservationRegistry` exists, so they are in the tree of the route spans;
   - GenAI spans are made current for synchronous calls (new default 
`Span.openScope()` in camel-telemetry), so the HTTP client and AI framework 
spans nest under them;
   - `camel-observability-services` includes `camel-ai-observability`.
   
   ### Points for review
   - `camel-telemetry`: `Span` gets default methods `setTag(String, long)`, 
`setTag(String, double)` and `openScope()`; only `camel-opentelemetry2` 
overrides them.
   - Reverses 
`GenAiMicrometerObservationTest.shouldPreferObservationOverTelemetrySpan` 
(CAMEL-24561): with the Micrometer path, Spring Boot with 
`camel-observability-services-starter` recorded no GenAI span at all, and the 
usage was missing.
   - `AgentInvocationListeners` (agent API) passes Camel's LangChain4j 
`AiServiceListener`s to the AI service built for the current invocation, 
without touching the user's `AgentConfiguration`. A custom `Agent` that does 
not use `AbstractAgent` keeps one generation for the run.
   - Spring AI models keep their `ObservationRegistry` in a private field; it 
is read by type, and an unreadable field means "not observed".
   - The behavior changes are in the 4.24 upgrade guide.
   
   ### Tests
   - Unit tests for the levels, the messages, the deferral (Spring AI, 
LangChain4j listener, OTel agent settings), the agent run tree, the openai tool 
loop with a mocked MCP tool, the Micrometer names and the Camel Main options. 
All unit tests of the touched modules pass.
   - Checked end to end with a self-hosted Langfuse on Camel Main, Spring Boot 
4.1 and Quarkus 4 (with the camel-spring-boot and camel-quarkus follow-ups 
below).
   
   ### Follow-ups
   - camel-spring-boot: 
[CAMEL-27133](https://issues.apache.org/jira/browse/CAMEL-27133) (starter 
options, implementation, observability-services), 
[CAMEL-27134](https://issues.apache.org/jira/browse/CAMEL-27134) (platform-http 
context propagation), 
[CAMEL-27132](https://issues.apache.org/jira/browse/CAMEL-27132) (thread-local 
starter options, a bug of today)
   - camel-quarkus: apache/camel-quarkus#9518 (new `ai-observability` extension)
   
   _Claude Code on behalf of Croway_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to