davsclaus opened a new pull request, #26548: URL: https://github.com/apache/camel/pull/26548
Resolves [CAMEL-24794](https://issues.apache.org/jira/browse/CAMEL-24794). ## Ollama tab A new tab in the More menu (AI group) that shows how the model served by Ollama is performing, in the spirit of the [llm-visuals](https://github.com/DingoOz/llm-visuals) dashboard but built on what Ollama actually exposes. It is listed only while an Ollama server answers (a background probe every ten seconds) and works with or without a running integration. ``` ╭ Ollama 0.33.3 · localhost:11434 · runner llama-server :58792 ─────────────────────────────────────────╮ │ qwen3.6:35b-a3b qwen35moe · 35.5B · Q4_K_M · 41 layers · 256 experts (8 active) · vision, tools, thinking │ 20.9G loaded, 100% GPU · ctx 32,768 of 262,144 · unloads in 29m11s ├ Throughput ────────────────────┬ Context ─────────────────────────────┬ Host ─────────────────────────┤ │ decode 52 tok/s last │ ▓▓▓▓▓▓▓▓░░░░░░░░ 11k / 32k (35%) │ GPU ▓░░░░░░░░░░ 7% │ │ prefill 554 tok/s │ prompt 11k · cache hit 86% (last) │ GPU mem 23.5G / 25.8G │ │ TTFT 2.8s · load 0s warm │ ○ idle · speculative draft-mtp │ llama-server 0.9% cpu · 27.2G│ │ session 56 tok/s · in 123k … │ turns ▁▁▁▁▂▂▂▂▂▂▂▂▂▂▂▃ 34% │ ollama serve 0.0% cpu · 58M │ ├ Requests (2 questions, 16 requests) ──────────────────────────────────────────────────────────────────┤ │ TIME SOURCE QUESTION IN OUT CACHE CTX PREFILL DECODE TTFT TOTAL │ │ ▶12:59:53 #2 ×3 tell about the timer 11k 482 92% 34% 515 53 0.57s 22s │ │ ▶12:59:00 #1 ×13 what DSL is this route using 9.0k 649 90% 27% 562 58 6.2s 25s │ ``` **Data sources, all discovered automatically** - Ollama API: version, loaded models (`/api/ps`: size, GPU share, allocated context, unload time), installed models, model shape from `/api/show` (architecture, layers, experts and active experts, max context, capabilities). Works against localhost, `camel infra run ollama` or a remote host. - Per-request timings Ollama returns: prompt and generation token counts and durations, load time, total time and cached prompt tokens. From them: prefill and decode tokens per second, time to first token, cache hit ratio, cold versus warm starts. The AI panel feeds these for its own requests; calls made by Camel routes arrive through the GenAI observability spans (tokens and duration, tagged with the route id). - Same machine only: Ollama's `llama-server` runner is found by its command line and its `/slots` endpoint polled for live state (working or idle, prompt tokens processed, cache tokens, tokens decoded, context size, speculative decoding), which gives a live decode rate and context fill for every caller. GPU utilization and memory via `ioreg` on macOS and `nvidia-smi` on Linux; CPU and memory of the server and runner via `ps` (the JDK exposes no CPU time for other processes on macOS). Remote or containerised Ollama keeps the API and per-request data and says which panels are unavailable. **Request log grouped per question.** A question asked in the AI panel costs one request per tool-call step, so the log shows one line per question with the question text, the number of requests it took (`×N`), the largest prompt, generated tokens, cache hit across the steps, the highest context fill, prefill and decode rates, time to the first token and the wall time from the first request to the last. Enter unfolds the steps. Route calls are one line each. **Context fill and compactions.** Each request carries the context window that served it; a CTX column and a per-turn "turns" line show how much of the window the prompt filled, with the session peak and the number of compactions (a prompt a fifth smaller than the previous turn). **MCP.** The same data is exposed as `tui_get_ollama` and through `tui_get_table` on the tab, including a `questions` array with the per-question aggregates. ## Other changes in this PR - `LlmClient.TokenUsage` (camel-jbang-core) keeps Ollama's `load_duration`, `total_duration` and `prompt_eval_cached_count`. Additive; the existing constructors remain. - **Bug fix, all tabs:** F1 help files were loaded through the classloader of a camel-jbang-core class. The `camel` launcher loads plugin jars in a child URLClassLoader with core in the parent, so no help file in the plugin jar was reachable: no F1 hint in the footer and F1 did nothing on any tab under the launcher (it worked on a flat classpath such as the tests). `DocHelper` now resolves from its own class first, like `Theme` does, with the context and core loaders as fallbacks. - `MoreTab` gains an `availableWhen` supplier that, unlike `activeWhen`, also applies when no integration is selected. ## Docs - User manual: the Ollama tab section, the More menu AI group, `tui_get_ollama` in the agent tools, and a new subsection **Working with a local Ollama model** under the AI provider docs: how a question is spent with Ollama (load, prefill, decode, time to first token, with figures measured on an M4 Pro with qwen3.6:35b-a3b), why one question costs several requests and how the prompt cache keeps that affordable, the 32k context window, compaction and its re-prefill cost, keeping every Ollama client on the same context size, model choice, where to look, remote and containerised Ollama, and links to the local model benchmark and GenAI observability blog posts. - F1 help for the tab (`tui/help/ollama.md`) explaining the phases, every panel and every column. ## Out of scope Speculative decoding acceptance rate and expert routing: they need `llama-server --metrics` or a llama.cpp patch, neither available through Ollama. Measured compaction and the context size default for the AI panel are tracked in [CAMEL-24796](https://issues.apache.org/jira/browse/CAMEL-24796), found with this tab. ## Testing - New tests: `OllamaParsersTest` (samples captured from Ollama 0.33.3 and its runner: `/api/ps`, `/api/show`, `/slots`, the runner command line, `ioreg`, `nvidia-smi`, `ps`), `TokenRateWindowTest`, `OllamaMonitorTest` (timings, cold starts, cache accounting, grouping per question, span ingestion, JSON), `OllamaTabRenderTest` (local and remote layouts, grouping and unfolding, trend line, help); `TabRegistryTest` covers the availability rule. - Full `camel-jbang-plugin-tui` suite: 1343 tests pass. - Verified live over MCP against Ollama 0.33.3 serving qwen3.6:35b-a3b on an Apple M4 Pro: runner discovery on its random port, live decode rate while generating, GPU and process load, AI panel questions landing in the log with Ollama's timings and grouped per question, route spans, the cached-token accounting cross-checked against the runner's own slot figures. _Claude Code on behalf of davsclaus_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
