[
https://issues.apache.org/jira/browse/CAMEL-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-27135:
--------------------------------
Summary: camel-jbang - show the agent runs, tool calls and messages of the
detailed and full GenAI observability levels (was: camel-jbang - camel monitor
shows the agent runs, tool calls and messages of the detailed and full GenAI
observability levels)
> camel-jbang - show the agent runs, tool calls and messages of the detailed
> and full GenAI observability levels
> --------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-27135
> URL: https://issues.apache.org/jira/browse/CAMEL-27135
> Project: Camel
> Issue Type: Sub-task
> Components: camel-jbang
> Reporter: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> CAMEL-27131 (PR https://github.com/apache/camel/pull/30291) adds the
> observability levels {{camel.aiObservability.level}} = {{basic}} (default) |
> {{detailed}} | {{full}}. At {{detailed}}, an agent run is an {{invoke_agent}}
> span with one {{chat}} span per model call and one {{execute_tool}} span per
> tool call, plus cached tokens, {{gen_ai.provider.name}} and
> {{gen_ai.conversation.id}}. At {{full}}, the spans also carry
> {{gen_ai.input.messages}}, {{gen_ai.output.messages}},
> {{gen_ai.system_instructions}} and the tool arguments and results.
> camel-jbang's monitor already reads the GenAI spans from the span dump of the
> running integration: the AI usage view and the Ollama view through
> {{GenAiSpanUsageExtractor}}, and the Spans view (tree plus every attribute).
> This ticket makes it use the new data.
> *1. Count only model calls (small fix, needed once CAMEL-27131 is merged)*
> {{GenAiSpanUsageExtractor.isGenAiSpan}} takes every span with
> {{gen_ai.operation.name}} as one model call. At {{detailed}} the
> {{invoke_agent}} and {{execute_tool}} spans would be listed as calls with 0
> tokens, and with the agent or tool name in place of the model. The token
> totals stay right, because CAMEL-27131 keeps the usage off the agent span.
> Count only the model operations (chat, generate_content, embeddings, ...) in
> the usage and Ollama views.
> *2. Show the messages as a conversation*
> At {{full}}, the messages are JSON strings of up to 8192 characters per text
> (the default {{contentMaxLength}}), shown as one line in the span details.
> Render them as a transcript: the system instructions, the user and assistant
> messages, and each tool call with its arguments and result.
> *3. Agent runs*
> Group a run ({{invoke_agent}}) with its model calls and tool calls, with the
> tokens and the time of each step, the cached tokens and the conversation id.
> Make it clear when a level below {{detailed}} is in use, so nothing seems
> missing.
> *Also*
> * Calls traced by an AI framework
> ({{camel.aiObservability.deferToFramework}}, default true) get no Camel span.
> That does not apply to a plain {{camel run}}, but the views should not assume
> a Camel GenAI span for every call.
> * The MCP server tools that report AI activity could add the agent-run steps
> from the same spans.
> The work goes on main only, after CAMEL-27131 is merged.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)