[
https://issues.apache.org/jira/browse/CAMEL-24805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116482#comment-18116482
]
Claus Ibsen commented on CAMEL-24805:
-------------------------------------
PR https://github.com/apache/camel/pull/26567: AnswerChecks in camel-jbang-core
extracts and validates the simple expressions of an answer (placeholders in
text and non-YAML blocks, only those starting with a simple function or value;
YAML blocks through SimpleChecks), and the AI panel sends an invalid answer
back once with what is wrong and the rule, shows the corrected answer with '1
correction' in the byline, or the unfixed answer with a Simple check line under
it.
_Claude Code on behalf of davsclaus_
> camel-jbang - TUI AI panel checks the simple expressions in an answer before
> showing it, and gives the model one turn to fix them
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24805
> URL: https://issues.apache.org/jira/browse/CAMEL-24805
> Project: Camel
> Issue Type: Improvement
> Components: camel-knative
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
>
> The TUI AI panel with a local Ollama model gets simple wrong in a specific,
> recurring way: given the catalog example {{$\{header.username\} ?: 'Guest'}}
> the model still writes {{$\{header.username ?: 'Guest'\}}} (the operator
> inside the placeholder) in every example, and similar slips with {{==}} and
> {{&&}} inside {{$\{...\}}}. A file the model writes goes through
> {{camel_write_file}}, which validates it and refuses an invalid one, so a
> route file ends up right after a retry. But the answer text does not: a
> snippet shown in the chat, or an expression in the explanation, reaches the
> user unchecked, and it is what the user copies.
> Proposed:
> * after the model's final answer, before it is rendered, extract the simple
> expressions from it: the {{$\{...\}}} placeholders and the fenced code blocks
> that hold a simple expression or a YAML step with {{simple:}};
> * run them through the checks the tools already use ({{SimpleChecks}} in
> camel-jbang-core, which is what {{camel_validate_source}} runs, plus the
> catalog's simple validator; {{SimpleChecks.hasPlaceholderAsLogicalOperand}}
> already detects the operator-inside-the-placeholder case);
> * when an error is found, send it back to the model as one correction turn
> ("the answer contains an invalid simple expression: ...; fix the answer")
> within the existing tool-call budget ({{AiPanel.MAX_ITERATIONS}}), and if the
> model does not fix it, show the answer with the validator's message below it
> so the user is not misled;
> * count the correction in the answer byline (CAMEL-24796) so it is measurable
> how often a model needs it.
> Out of scope: the external MCP server ({{camel mcp}}); there the client
> renders the answer and the tools already validate what is written.
> Tests in {{AiPanelTest}} with the fake client: an answer with {{$\{header.foo
> ?: 'x'\}}} triggers one correction turn; a corrected answer is shown as is;
> an answer that stays wrong is shown with the message.
> Context: the simple support for local models in the TUI AI panel (Sept 2026,
> {{camel_eval_expression}}, the simple primer in the system prompt and the
> function groups of {{camel_catalog_doc}}); this is the last step, checking
> what the model says rather than only what it writes.
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
