[
https://issues.apache.org/jira/browse/CAMEL-27313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18148572#comment-18148572
]
Claus Ibsen commented on CAMEL-27313:
-------------------------------------
Moved to 4.24.0. The design:
* *Where the data comes from:* the {{GenAiObservation}} calls the producers
already make (usage, cached tokens, messages, tool arguments and results). The
console is one more recorder next to the metrics and the spans, so it needs no
tracer and no {{--observe}}. It records only when the dev console is enabled
({{camel.main.devConsoleEnabled}}, on in the dev profile), so production pays
nothing.
* *Nesting:* a model or tool call goes under
{{GenAiObservationContext.parent()}}, else under the agent run or tool call
open on the exchange. The recorder keeps its own stack on the exchange, as the
span stack does, so camel-openai agent runs nest too.
* *What it keeps:*
** totals since start: per model, the calls, tokens in, out and cached, errors
and time; plus the agent runs and tool calls
** a bounded buffer of the newest top-level calls (agent runs, and model calls
outside a run), each with its steps in order
* *The level is stated, not guessed:* the console knows
{{camel.aiObservability.level}}.
* *Content:* messages, system instructions and tool arguments and results only
at level {{full}}, capped by {{camel.aiObservability.contentMaxLength}}, and
only in the answer when asked ({{content=true}}, or one run by {{id}}).
* *The clients* (the CLI, the TUI, the MCP server, Kaoto) read the console.
There is no fallback to the span dump for older Camel versions: this is for new
versions. The span views of the TUI (the waterfall and the conversation of a
trace) stay, as they show one trace.
_Claude Code on behalf of Claus Ibsen_
> camel-ai-observability - a genai dev console with the agent runs, model calls
> and tool calls
> --------------------------------------------------------------------------------------------
>
> Key: CAMEL-27313
> URL: https://issues.apache.org/jira/browse/CAMEL-27313
> Project: Camel
> Issue Type: Sub-task
> Components: camel-ai
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> The GenAI observability (CAMEL-27131) records model calls, agent runs, tool
> calls and, at level {{full}}, the messages as OpenTelemetry spans. Today
> every client works them out from the raw span dump:
> * the Camel TUI: CAMEL-27135
> * the Camel MCP server: CAMEL-27312, through a span model shared in
> camel-jbang-core
> A {{genai}} dev console in camel-ai-observability would give every client the
> same JSON straight from the runtime, Kaoto and other tools included:
> * the model calls per model (tokens in, out and cached, latency, finish
> reasons)
> * the agent runs with their steps in order
> * the level in use
> * on request, the conversation of one run
> It would also work without a tracer, if the console keeps its own bounded
> buffer.
> The console would carry the same rules as the views it serves. Only model
> operations count as model calls. A run is matched by its spans (or
> observations). Message content is shown only at level {{full}}, only when
> asked, and capped, because it can hold customers' data.
> The CLI, the TUI and the MCP server (CAMEL-27312) could then read the console
> instead of the span dump.
> Suggested by the local model benchmark work, planned by Claus for 4.25.0.
> _Created by Claude Code on behalf of Claus Ibsen_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)