This is an automated email from the ASF dual-hosted git repository. davsclaus pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/camel.git
commit d202eff67b1745427de78452f777e3323bf944a1 Author: Claus Ibsen <[email protected]> AuthorDate: Tue Sep 8 16:05:40 2026 +0200 Camel TUI: Recommend qwen3.6:35b-a3b as the local Ollama model for the AI panel The F8 panel sends a tool-heavy prompt of ~8k tokens on every question, so prompt processing dominates latency with local models. A dense 27B/32B model prefills at ~110 tok/s on an M4 Pro (about a minute per question), while the mixture-of-experts qwen3.6:35b-a3b with 3B active parameters prefills at ~750 tok/s and decodes faster too, so answers start in seconds. Update the F2 Setup AI page, the F8 getting-started guide, the doctor hint, the model auto-selection preference, and the TUI and AI user-manual pages to recommend qwen3.6:35b-a3b, and note that on Apple Silicon the default GGUF tags should be used because the Ollama MLX engine cannot yet reuse the cached prompt for Qwen 3.x models. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01DHRH7agh6Fh1ExD9e261H6 Signed-off-by: Claus Ibsen <[email protected]> --- .../modules/ROOT/pages/camel-jbang-ai.adoc | 15 ++++++++++----- .../modules/ROOT/pages/camel-jbang-tui.adoc | 17 ++++++++++++----- .../camel/dsl/jbang/core/commands/LlmClient.java | 2 +- .../camel/dsl/jbang/core/commands/tui/AiPanel.java | 14 ++++++++------ .../dsl/jbang/core/commands/tui/DocViewerPopup.java | 19 +++++++++++++------ .../dsl/jbang/core/commands/tui/DoctorPopup.java | 2 +- 6 files changed, 45 insertions(+), 24 deletions(-) diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc index baa4f1e33852..74f11eecd891 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc @@ -148,7 +148,7 @@ brew install ollama curl -fsSL https://ollama.com/install.sh | sh # Pull a model and start asking -ollama pull qwen2.5:32b +ollama pull qwen3.6:35b-a3b camel ask "what routes are running?" ---- @@ -159,19 +159,24 @@ The CLI checks what models are available and auto-selects a suitable one. `camel ask` and the TUI F8 panel rely on tool calling to inspect your running Camel process. Models smaller than ~14B do not reliably invoke tools and answer from training knowledge instead. -Use at least a 14B model; 32B is recommended. +Use at least a 14B model. Prefer a mixture-of-experts model such as `qwen3.6:35b-a3b`: with only +3B parameters active per token it processes the tool-heavy prompt many times faster than a dense +27B/32B model, so answers start in seconds instead of a minute. [options="header"] |=== | Model | RAM (Q4) | Notes -| `qwen2.5:14b` | ~9 GB | Minimum recommended -| `qwen2.5:32b` | ~20 GB | Best balance of speed and quality -| `deepseek-r1:32b` | ~20 GB | Strong reasoning +| `qwen3.6:35b-a3b` | ~23 GB | Recommended: fastest prompt processing, needs 32 GB+ +| `qwen2.5:14b` | ~9 GB | Minimum for 16 GB machines +| `qwen3.6:27b` | ~18 GB | Strong dense model, several times slower prompt processing +| `qwen2.5:32b` | ~20 GB | Good quality, slow prompt processing | `hermes3:70b` | ~43 GB | Excellent tool calling, needs 64 GB+ | `llama3.3:70b` | ~43 GB | Best open model, needs 64 GB+ |=== On Apple Silicon, all RAM is unified — a 64 GB M-series Mac can run `llama3.3:70b` comfortably alongside the OS and other dev tools. +Use the default (GGUF) tags rather than the `-mlx` tags: the Ollama MLX engine cannot yet reuse the +cached prompt for Qwen 3.x models, so every question re-processes the whole prompt. === Using an OpenAI-compatible local server diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc index 43b25261142f..90c4d0b0f1ed 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc @@ -782,7 +782,7 @@ brew install ollama curl -fsSL https://ollama.com/install.sh | sh # Pull a model — then open the TUI and press F8 -ollama pull qwen2.5:32b +ollama pull qwen3.6:35b-a3b camel tui ---- @@ -790,16 +790,19 @@ Ollama at `localhost:11434` is auto-detected. No configuration needed. IMPORTANT: The F8 AI panel works by invoking built-in tools to inspect your running Camel process. Models smaller than ~14B do not reliably call tools and answer from training knowledge instead. -Use at least a 14B model; 32B is recommended. +Use at least a 14B model. Prefer a mixture-of-experts model such as `qwen3.6:35b-a3b`: with only +3B parameters active per token it processes the tool-heavy prompt many times faster than a dense +27B/32B model, so answers start in seconds instead of a minute. *Models that work well* (tool-calling capable, ≥14B, default Q4_K_M quantization): [options="header"] |=== | Model | RAM | Notes -| `qwen2.5:14b` | ~9 GB | Minimum recommended -| `qwen2.5:32b` | ~20 GB | Best balance of speed and quality -| `deepseek-r1:32b` | ~20 GB | Strong reasoning +| `qwen3.6:35b-a3b` | ~23 GB | Recommended: fastest prompt processing, needs 32 GB+ +| `qwen2.5:14b` | ~9 GB | Minimum for 16 GB machines +| `qwen3.6:27b` | ~18 GB | Strong dense model, several times slower prompt processing +| `qwen2.5:32b` | ~20 GB | Good quality, slow prompt processing | `hermes3:70b` | ~43 GB | Excellent tool calling, needs 64 GB+ | `llama3.3:70b` | ~43 GB | Best open model, needs 64 GB+ |=== @@ -807,6 +810,10 @@ Use at least a 14B model; 32B is recommended. NOTE: `camel infra run ollama` runs Ollama in Docker and bypasses GPU acceleration, making inference significantly slower. Native install is preferred for development. +NOTE: On Apple Silicon, use the default (GGUF) tags rather than the `-mlx` tags. The Ollama MLX +engine cannot yet reuse the cached prompt for Qwen 3.x models, so every question re-processes the +whole prompt, while the default engine reuses it and only processes what is new. + ==== Using an OpenAI-compatible local server Set `LLM_API_KEY` and `LLM_BASE_URL` to connect to any OpenAI-compatible server diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java index 4ada1d9fb0c6..1d8470f287e7 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java @@ -1965,7 +1965,7 @@ public class LlmClient { } List<String> preferred - = List.of("qwen3.5", "qwen3", "nemotron-3-nano", "mistral-nemo", + = List.of("qwen3.6", "qwen3.5", "qwen3", "nemotron-3-nano", "mistral-nemo", "qwen2.5", "granite4.1", "llama3.1", "llama3.3", "mistral"); for (String pref : preferred) { for (String avail : available) { diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java index 0a0e15112ba6..19caf4aecfd1 100644 --- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java +++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java @@ -1595,8 +1595,9 @@ class AiPanel { > **Tool calling is required.** This panel inspects your Camel process by > invoking built-in tools. Models smaller than ~14B do not reliably call - > tools and will answer from training knowledge instead — use at least 14B, - > 32B recommended. + > tools and will answer from training knowledge instead — use at least 14B. + > Prefer a mixture-of-experts model such as qwen3.6:35b-a3b: it processes + > the tool-heavy prompt many times faster than a dense 27B/32B model. --- @@ -1613,7 +1614,7 @@ class AiPanel { # then on both: ollama serve # start the daemon (skip if auto-started) - ollama pull qwen2.5:32b # recommended + ollama pull qwen3.6:35b-a3b # recommended ``` Ollama is auto-detected at `localhost:11434` — no configuration needed. @@ -1622,9 +1623,10 @@ class AiPanel { | Model | RAM | Notes | |---|---|---| - | qwen2.5:14b | ~9 GB | Minimum recommended | - | qwen2.5:32b | ~20 GB | Best balance of speed and quality | - | deepseek-r1:32b | ~20 GB | Strong reasoning | + | qwen3.6:35b-a3b | ~23 GB | Recommended: only 3B active per token, fastest prompt processing | + | qwen2.5:14b | ~9 GB | Minimum for 16 GB machines | + | qwen3.6:27b | ~18 GB | Strong dense model, several times slower prompt processing | + | qwen2.5:32b | ~20 GB | Good quality, slow prompt processing | | hermes3:70b | ~43 GB | Excellent tool calling, needs 64 GB+ | | llama3.3:70b | ~43 GB | Best open model, needs 64 GB+ | diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java index ce1f0c11de77..2a75a6523991 100644 --- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java +++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java @@ -390,7 +390,7 @@ class DocViewerPopup { status = "**Detected:** Ollama at " + OllamaDoctorSupport.formatDisplayHost(ollama.baseUrl()) + " with " + OllamaDoctorSupport.modelCountLabel(ollama.models()); } else if (ollama.running()) { - status = "**Detected:** Ollama is running but has no models. Run `ollama pull qwen2.5:32b`."; + status = "**Detected:** Ollama is running but has no models. Run `ollama pull qwen3.6:35b-a3b`."; } else { status = "**Status:** No AI provider detected. Set an API key or start Ollama, then press F8."; } @@ -422,18 +422,25 @@ class DocViewerPopup { + "Install Ollama natively for GPU acceleration, pull a model, then press F8:\n\n" + " brew install ollama # macOS\n" + " curl -fsSL https://ollama.com/install.sh | sh # Linux\n" - + " ollama pull qwen2.5:32b\n\n" + + " ollama pull qwen3.6:35b-a3b\n\n" + "The AI panel relies on tool calling. Models smaller than ~14B do not call tools\n" - + "reliably and answer from training data instead. Use at least 14B; 32B is recommended.\n\n" + + "reliably and answer from training data instead. Use at least 14B. Prefer a\n" + + "mixture-of-experts model such as `qwen3.6:35b-a3b`: with only 3B parameters active\n" + + "per token it processes the tool-heavy prompt many times faster than a dense 27B/32B\n" + + "model, so answers start in seconds instead of a minute.\n\n" + "| Model | RAM | Notes |\n" + "|-------|-----|-------|\n" - + "| `qwen2.5:14b` | ~9 GB | Minimum recommended |\n" - + "| `qwen2.5:32b` | ~20 GB | Best balance of speed and quality |\n" - + "| `deepseek-r1:32b` | ~20 GB | Strong reasoning |\n" + + "| `qwen3.6:35b-a3b` | ~23 GB | Recommended: fastest prompt processing, needs 32 GB+ |\n" + + "| `qwen2.5:14b` | ~9 GB | Minimum for 16 GB machines |\n" + + "| `qwen3.6:27b` | ~18 GB | Strong dense model, several times slower prompt processing |\n" + + "| `qwen2.5:32b` | ~20 GB | Good quality, slow prompt processing |\n" + "| `hermes3:70b` | ~43 GB | Excellent tool calling, needs 64 GB+ |\n" + "| `llama3.3:70b` | ~43 GB | Best open model, needs 64 GB+ |\n\n" + "`camel infra run ollama` runs Ollama in Docker without GPU acceleration, which is\n" + "much slower. Prefer the native install for development.\n\n" + + "On Apple Silicon, use the default (GGUF) tags rather than the `-mlx` tags: the Ollama\n" + + "MLX engine cannot yet reuse the cached prompt for Qwen 3.x models, so every question\n" + + "re-processes the whole prompt.\n\n" + "## 3. OpenAI-compatible local servers\n\n" + "LM Studio, vLLM, llama.cpp, GPT4All and similar servers work through `LLM_BASE_URL`:\n\n" + " export LLM_API_KEY=any-value\n" diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java index 048a90839c1a..bb33273a5362 100644 --- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java +++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java @@ -397,7 +397,7 @@ class DoctorPopup { Style.EMPTY.dim()))); if (allSmall) { result.add(Line.from(Span.styled( - " F8 needs ≥14B — run: ollama pull qwen2.5:14b", + " F8 needs ≥14B — run: ollama pull qwen3.6:35b-a3b", Style.EMPTY.dim()))); } } else {
