davsclaus opened a new pull request, #26550: URL: https://github.com/apache/camel/pull/26550
JIRA: https://issues.apache.org/jira/browse/CAMEL-24796 Follow-up to the Ollama tab (CAMEL-24794, #26548). With a local model the AI panel compacted its history from a character estimate, asked Ollama for a 32k window when the model could hold more, and gave no sign of what a question had cost or that the tool-call limit had ended it. This PR sizes the context for the machine, compacts on what Ollama measured, and makes the cost of a question visible in the answer, in the Ollama tab and over MCP. ## Context window and compaction - The panel asks Ollama for a window by this rule: `OLLAMA_CONTEXT_LENGTH` when set, else the window already loaded (raised to 32k), else 64k when the weights plus the KV cache for 64k fit in four fifths of the machine's memory less 1 GiB, else 32k. The KV cost per token comes from `/api/show` (layers, full-attention interval, KV heads, key and value length), so a hybrid-attention model such as `qwen3.6:35b-a3b` gets 64k on a 64 GB Mac and 32k on 32 GB. - Compaction uses the prompt size Ollama reported for the last request, not a character estimate. A local endpoint compacts once the measured prompt passes half the window (capped at 64k), and compacts hard, down to a quarter of the window, so it does not re-trigger every turn. Hosted endpoints keep their light per-turn compaction. - `/compact` and the automatic compaction print one conversation line saying what was dropped and how long the next reply will take to start, instead of a dialog. `/context` shows the window, the compaction point and the last measured prompt; the panel title shows `ctx N%`. ## What a question cost - Every answer ends with a byline, for example `61.1s · 25 tool calls, limit reached (2.5s in tools) · 26 requests · 231.3k tokens · ctx 18%`. - When the 25 tool-call limit is reached the wrap-up request keeps the tool definitions in the prompt, so Ollama's prompt cache holds; a model that calls a tool regardless is asked once more without tools. Both are recorded with the reason `limit`. - AI Usage (Ctrl+U) says `Avg per question: 5.5s (2.3s per request)` and the model table column is `AVG/REQ`, so it agrees with the Ollama tab, which is per question. ## Ollama tab - Header separates the model's maximum context (`max ctx 256k`) from the loaded window (`ctx 64k`) and shows where the AI panel compacts (`AI panel compacts above 32k`) and what it asks for when that differs. Token counts read `64k`, no thousands separators. - The question the AI panel is working on is listed from the moment it is asked, with `working` and a spinner as its reason and TOTAL counting up; the title says `1 in progress`. - A footer row, `avg/question`, gives the session's average per question for IN, OUT, TTFT and TOTAL, the pooled cache hit and rates, the peak CTX and how many questions ended at the tool-call limit. Route calls are left out. - Costly questions are coloured: the `×N` request count yellow from 10 and red when the question ended at the limit; TOTAL yellow from 30 s and orange from a minute; `limit` in yellow in the REASON column. - A question's wall time starts at its first request. A smaller prompt inside one question (the wrap-up) is no longer counted as a compaction. - The runner's `/slots` is read twice a second while generating and every two seconds when idle. - `tui_get_ollama` carries all of it: `aiPanel` (window, compaction point), `inProgress` and `elapsedMs` on the active question, and a `questionSummary` object. ## Docs and tests - TUI page: Ollama section, "Working with a local Ollama model", the AI Usage paragraph and the slash command rows. AI commands page: new "Context window and keep-alive" section. Upgrade guide 4.23: the TUI entry describes the window policy. F1 help on the Ollama tab covers the header, the in-progress row, the footer, the colours and `limit`. - New and extended tests: `LlmClientOllamaContextTest` (window policy and KV formula against a fake Ollama), `AiPanelHistoryCompactionTest`, `AiPanelTest` (byline, wrap-up with tools and the fallback, usage averages), `OllamaMonitorTest`, `OllamaTabRenderTest`. The camel-jbang-plugin-tui suite passes (1368 tests). _Claude Code on behalf of davsclaus_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
