[ 
https://issues.apache.org/jira/browse/CAMEL-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claus Ibsen updated CAMEL-27135:
--------------------------------
    Summary: camel-jbang - show the agent runs, tool calls and messages of the 
detailed and full GenAI observability levels  (was: camel-jbang - camel monitor 
shows the agent runs, tool calls and messages of the detailed and full GenAI 
observability levels)

> camel-jbang - show the agent runs, tool calls and messages of the detailed 
> and full GenAI observability levels
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-27135
>                 URL: https://issues.apache.org/jira/browse/CAMEL-27135
>             Project: Camel
>          Issue Type: Sub-task
>          Components: camel-jbang
>            Reporter: Claus Ibsen
>            Priority: Major
>             Fix For: 4.24.0
>
>
> CAMEL-27131 (PR https://github.com/apache/camel/pull/30291) adds the 
> observability levels {{camel.aiObservability.level}} = {{basic}} (default) | 
> {{detailed}} | {{full}}. At {{detailed}}, an agent run is an {{invoke_agent}} 
> span with one {{chat}} span per model call and one {{execute_tool}} span per 
> tool call, plus cached tokens, {{gen_ai.provider.name}} and 
> {{gen_ai.conversation.id}}. At {{full}}, the spans also carry 
> {{gen_ai.input.messages}}, {{gen_ai.output.messages}}, 
> {{gen_ai.system_instructions}} and the tool arguments and results.
> camel-jbang's monitor already reads the GenAI spans from the span dump of the 
> running integration: the AI usage view and the Ollama view through 
> {{GenAiSpanUsageExtractor}}, and the Spans view (tree plus every attribute). 
> This ticket makes it use the new data.
> *1. Count only model calls (small fix, needed once CAMEL-27131 is merged)*
> {{GenAiSpanUsageExtractor.isGenAiSpan}} takes every span with 
> {{gen_ai.operation.name}} as one model call. At {{detailed}} the 
> {{invoke_agent}} and {{execute_tool}} spans would be listed as calls with 0 
> tokens, and with the agent or tool name in place of the model. The token 
> totals stay right, because CAMEL-27131 keeps the usage off the agent span. 
> Count only the model operations (chat, generate_content, embeddings, ...) in 
> the usage and Ollama views.
> *2. Show the messages as a conversation*
> At {{full}}, the messages are JSON strings of up to 8192 characters per text 
> (the default {{contentMaxLength}}), shown as one line in the span details. 
> Render them as a transcript: the system instructions, the user and assistant 
> messages, and each tool call with its arguments and result.
> *3. Agent runs*
> Group a run ({{invoke_agent}}) with its model calls and tool calls, with the 
> tokens and the time of each step, the cached tokens and the conversation id. 
> Make it clear when a level below {{detailed}} is in use, so nothing seems 
> missing.
> *Also*
> * Calls traced by an AI framework 
> ({{camel.aiObservability.deferToFramework}}, default true) get no Camel span. 
> That does not apply to a plain {{camel run}}, but the views should not assume 
> a Camel GenAI span for every call.
> * The MCP server tools that report AI activity could add the agent-run steps 
> from the same spans.
> The work goes on main only, after CAMEL-27131 is merged.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to