[
https://issues.apache.org/jira/browse/CAMEL-24805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-24805 started by Claus Ibsen.
-------------------------------------------
> camel-jbang - TUI AI panel checks the simple expressions in an answer before
> showing it, and gives the model one turn to fix them
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24805
> URL: https://issues.apache.org/jira/browse/CAMEL-24805
> Project: Camel
> Issue Type: Improvement
> Components: camel-knative
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
>
> The TUI AI panel with a local Ollama model gets simple wrong in a specific,
> recurring way: given the catalog example {{$\{header.username\} ?: 'Guest'}}
> the model still writes {{$\{header.username ?: 'Guest'\}}} (the operator
> inside the placeholder) in every example, and similar slips with {{==}} and
> {{&&}} inside {{$\{...\}}}. A file the model writes goes through
> {{camel_write_file}}, which validates it and refuses an invalid one, so a
> route file ends up right after a retry. But the answer text does not: a
> snippet shown in the chat, or an expression in the explanation, reaches the
> user unchecked, and it is what the user copies.
> Proposed:
> * after the model's final answer, before it is rendered, extract the simple
> expressions from it: the {{$\{...\}}} placeholders and the fenced code blocks
> that hold a simple expression or a YAML step with {{simple:}};
> * run them through the checks the tools already use ({{SimpleChecks}} in
> camel-jbang-core, which is what {{camel_validate_source}} runs, plus the
> catalog's simple validator; {{SimpleChecks.hasPlaceholderAsLogicalOperand}}
> already detects the operator-inside-the-placeholder case);
> * when an error is found, send it back to the model as one correction turn
> ("the answer contains an invalid simple expression: ...; fix the answer")
> within the existing tool-call budget ({{AiPanel.MAX_ITERATIONS}}), and if the
> model does not fix it, show the answer with the validator's message below it
> so the user is not misled;
> * count the correction in the answer byline (CAMEL-24796) so it is measurable
> how often a model needs it.
> Out of scope: the external MCP server ({{camel mcp}}); there the client
> renders the answer and the tools already validate what is written.
> Tests in {{AiPanelTest}} with the fake client: an answer with {{$\{header.foo
> ?: 'x'\}}} triggers one correction turn; a corrected answer is shown as is;
> an answer that stays wrong is shown with the message.
> Context: the simple support for local models in the TUI AI panel (Sept 2026,
> {{camel_eval_expression}}, the simple primer in the system prompt and the
> function groups of {{camel_catalog_doc}}); this is the last step, checking
> what the model says rather than only what it writes.
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
