[
https://issues.apache.org/jira/browse/CAMEL-27131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-27131 started by Federico Mariani.
------------------------------------------------
> camel-ai-observability - observability levels with agent, tool and message
> data, and no double recording of calls traced by AI frameworks
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-27131
> URL: https://issues.apache.org/jira/browse/CAMEL-27131
> Project: Camel
> Issue Type: Sub-task
> Components: camel-ai, camel-langchain4j-agent, camel-openai,
> camel-spring-ai, camel-telemetry
> Reporter: Federico Mariani
> Assignee: Federico Mariani
> Priority: Major
>
> Today the AI producers record one span per LLM call with the model, the token
> usage and the finish reason, the stable subset chosen in CAMEL-23861. Tracing
> backends built around LLM traces, which read the OpenTelemetry GenAI semantic
> conventions (for example Langfuse), need more to show what an AI route did,
> and several common setups record each call twice.
> Tested end to end with a self-hosted Langfuse on Camel Main, Spring Boot 4.1
> and Quarkus 4, with {{openai}}, {{langchain4j-agent}} and an {{ai-tool}}
> route, {{spring-ai-chat}} and {{langchain4j-chat}}:
> * no prompt or completion is recorded on any LLM call;
> * a {{langchain4j-agent}} run is one {{generate_content}} span that sums
> every model call of the tool loop, and the tool calls are not traced as tools;
> * the same call is recorded twice, with its tokens, by the OpenTelemetry Java
> agent (its {{openai-java}} instrumentation), by Spring AI ({{ChatModel}}
> observations) and by quarkus-langchain4j ({{SpanChatModelListener}});
> * with both {{camel-opentelemetry2}} and a Micrometer {{ObservationRegistry}}
> (Spring Boot), the Micrometer path is taken: it drops the token usage, the
> response model and the finish reason, and records nothing when no tracing
> handler is registered;
> * {{camel-observability-services}} does not include
> {{camel-ai-observability}}.
> Proposal, keeping today's output as the default:
> * {{camel.aiObservability.level}} = {{basic}} (default, unchanged) |
> {{detailed}} | {{full}}
> ** {{detailed}}: an {{invoke_agent}} span per {{langchain4j-agent}} run and
> per {{openai}} tool loop, one {{chat}} span per model call, an
> {{execute_tool}} span per tool call (Camel route tools through
> {{AiToolExecutor}}, MCP tools of {{openai}}), request parameters, cached
> tokens, {{gen_ai.provider.name}} and {{gen_ai.conversation.id}};
> ** {{full}}: {{gen_ai.input.messages}}, {{gen_ai.output.messages}},
> {{gen_ai.system_instructions}}, tool arguments and results, each text cut at
> {{camel.aiObservability.contentMaxLength}} (8192).
> * {{camel.aiObservability.deferToFramework}} (default true): no Camel span
> for the calls that Spring AI, quarkus-langchain4j, a
> {{langchain4j-observation}} listener or the OpenTelemetry Java agent already
> trace.
> * Metrics are the same at every level. Agent runs and tool calls recorded
> through Micrometer are named {{camel.ai.agent}} and {{camel.ai.tool}}, apart
> from {{gen_ai.client.operation}}.
> * At every level: numeric token attributes, the usage as high-cardinality
> values of the Micrometer Observation (as Spring AI does), GenAI spans through
> {{camel-opentelemetry2}} when it is in use, and GenAI spans made current
> during synchronous calls so that the spans of the HTTP client or the AI
> framework nest under them.
> * {{camel-observability-services}} depends on {{camel-ai-observability}}.
> The camel-spring-boot and camel-quarkus parts are tracked separately.
> _Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)