Claus Ibsen created CAMEL-27135:
-----------------------------------

             Summary: camel-jbang - camel monitor shows the agent runs, tool 
calls and messages of the detailed and full GenAI observability levels
                 Key: CAMEL-27135
                 URL: https://issues.apache.org/jira/browse/CAMEL-27135
             Project: Camel
          Issue Type: Sub-task
          Components: camel-jbang
            Reporter: Claus Ibsen
             Fix For: 4.24.0


CAMEL-27131 (PR https://github.com/apache/camel/pull/30291) adds the 
observability levels {{camel.aiObservability.level}} = {{basic}} (default) | 
{{detailed}} | {{full}}. At {{detailed}}, an agent run is an {{invoke_agent}} 
span with one {{chat}} span per model call and one {{execute_tool}} span per 
tool call, plus cached tokens, {{gen_ai.provider.name}} and 
{{gen_ai.conversation.id}}. At {{full}}, the spans also carry 
{{gen_ai.input.messages}}, {{gen_ai.output.messages}}, 
{{gen_ai.system_instructions}} and the tool arguments and results.

camel-jbang's monitor already reads the GenAI spans from the span dump of the 
running integration: the AI usage view and the Ollama view through 
{{GenAiSpanUsageExtractor}}, and the Spans view (tree plus every attribute). 
This ticket makes it use the new data.

*1. Count only model calls (small fix, needed once CAMEL-27131 is merged)*
{{GenAiSpanUsageExtractor.isGenAiSpan}} takes every span with 
{{gen_ai.operation.name}} as one model call. At {{detailed}} the 
{{invoke_agent}} and {{execute_tool}} spans would be listed as calls with 0 
tokens, and with the agent or tool name in place of the model. The token totals 
stay right, because CAMEL-27131 keeps the usage off the agent span. Count only 
the model operations (chat, generate_content, embeddings, ...) in the usage and 
Ollama views.

*2. Show the messages as a conversation*
At {{full}}, the messages are JSON strings of up to 8192 characters per text 
(the default {{contentMaxLength}}), shown as one line in the span details. 
Render them as a transcript: the system instructions, the user and assistant 
messages, and each tool call with its arguments and result.

*3. Agent runs*
Group a run ({{invoke_agent}}) with its model calls and tool calls, with the 
tokens and the time of each step, the cached tokens and the conversation id. 
Make it clear when a level below {{detailed}} is in use, so nothing seems 
missing.

*Also*
* Calls traced by an AI framework ({{camel.aiObservability.deferToFramework}}, 
default true) get no Camel span. That does not apply to a plain {{camel run}}, 
but the views should not assume a Camel GenAI span for every call.
* The MCP server tools that report AI activity could add the agent-run steps 
from the same spans.

The work goes on main only, after CAMEL-27131 is merged.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to