[ 
https://issues.apache.org/jira/browse/CAMEL-27131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Federico Mariani reassigned CAMEL-27131:
----------------------------------------

    Assignee: Federico Mariani

> camel-ai-observability - observability levels with agent, tool and message 
> data, and no double recording of calls traced by AI frameworks
> -----------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-27131
>                 URL: https://issues.apache.org/jira/browse/CAMEL-27131
>             Project: Camel
>          Issue Type: Sub-task
>          Components: camel-ai, camel-langchain4j-agent, camel-openai, 
> camel-spring-ai, camel-telemetry
>            Reporter: Federico Mariani
>            Assignee: Federico Mariani
>            Priority: Major
>
> Today the AI producers record one span per LLM call with the model, the token 
> usage and the finish reason, the stable subset chosen in CAMEL-23861. Tracing 
> backends built around LLM traces, which read the OpenTelemetry GenAI semantic 
> conventions (for example Langfuse), need more to show what an AI route did, 
> and several common setups record each call twice.
> Tested end to end with a self-hosted Langfuse on Camel Main, Spring Boot 4.1 
> and Quarkus 4, with {{openai}}, {{langchain4j-agent}} and an {{ai-tool}} 
> route, {{spring-ai-chat}} and {{langchain4j-chat}}:
> * no prompt or completion is recorded on any LLM call;
> * a {{langchain4j-agent}} run is one {{generate_content}} span that sums 
> every model call of the tool loop, and the tool calls are not traced as tools;
> * the same call is recorded twice, with its tokens, by the OpenTelemetry Java 
> agent (its {{openai-java}} instrumentation), by Spring AI ({{ChatModel}} 
> observations) and by quarkus-langchain4j ({{SpanChatModelListener}});
> * with both {{camel-opentelemetry2}} and a Micrometer {{ObservationRegistry}} 
> (Spring Boot), the Micrometer path is taken: it drops the token usage, the 
> response model and the finish reason, and records nothing when no tracing 
> handler is registered;
> * {{camel-observability-services}} does not include 
> {{camel-ai-observability}}.
> Proposal, keeping today's output as the default:
> * {{camel.aiObservability.level}} = {{basic}} (default, unchanged) | 
> {{detailed}} | {{full}}
> ** {{detailed}}: an {{invoke_agent}} span per {{langchain4j-agent}} run and 
> per {{openai}} tool loop, one {{chat}} span per model call, an 
> {{execute_tool}} span per tool call (Camel route tools through 
> {{AiToolExecutor}}, MCP tools of {{openai}}), request parameters, cached 
> tokens, {{gen_ai.provider.name}} and {{gen_ai.conversation.id}};
> ** {{full}}: {{gen_ai.input.messages}}, {{gen_ai.output.messages}}, 
> {{gen_ai.system_instructions}}, tool arguments and results, each text cut at 
> {{camel.aiObservability.contentMaxLength}} (8192).
> * {{camel.aiObservability.deferToFramework}} (default true): no Camel span 
> for the calls that Spring AI, quarkus-langchain4j, a 
> {{langchain4j-observation}} listener or the OpenTelemetry Java agent already 
> trace.
> * Metrics are the same at every level. Agent runs and tool calls recorded 
> through Micrometer are named {{camel.ai.agent}} and {{camel.ai.tool}}, apart 
> from {{gen_ai.client.operation}}.
> * At every level: numeric token attributes, the usage as high-cardinality 
> values of the Micrometer Observation (as Spring AI does), GenAI spans through 
> {{camel-opentelemetry2}} when it is in use, and GenAI spans made current 
> during synchronous calls so that the spans of the HTTP client or the AI 
> framework nest under them.
> * {{camel-observability-services}} depends on {{camel-ai-observability}}.
> The camel-spring-boot and camel-quarkus parts are tracked separately.
> _Claude Code on behalf of Croway_



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to