davsclaus opened a new pull request, #30145: URL: https://github.com/apache/camel/pull/30145
A follow-up to #29962 of [CAMEL-26932](https://issues.apache.org/jira/browse/CAMEL-26932). ## Problem Since #29962 a repeated catalog miss without suggestions gets the note "Ask with other arguments instead.". For `camel_catalog_sample` that note replaced the hint of CAMEL-26875, which was written for exactly this case: there is no sample, so write the file, whose checks say what to change. In the local-model benchmark of last night (r34), in a Citrus test step, the model asked `camel_catalog_sample` for `send request` 10 times. The answer was "No sample for 'send request'", and the last 8 answers were the note that said to ask with other arguments. The step failed. ## Change A tool can give a miss hint (`ToolDescriptor.missHint`). The note of a repeated miss without suggestions uses it instead of "Ask with other arguments instead.". `camel_catalog_sample` gives one: ``` "note": "The answer was: No sample for 'send request'. Write the file now with camel_write_file: asking again gives no sample either. The write checks the file, and its answer says what to change (for a Citrus test: which actions it takes). You asked this 3 times now with the same arguments, and the answer is the same each time." ``` Not changed: - a miss with suggestions ("Use one of the suggestions instead: ..."); - a miss of a tool without a miss hint, such as `camel_catalog_doc`, whose hint is for an answer that was found; - the note of a repeated answer that was found. This gives the miss the hint it was meant to have. It may not be enough on its own: in the same night the hint for answers that were found came up 79 times, and the model mostly asked again. ## Tests - `RepeatedToolCallsTest`: a repeated `camel_catalog_sample` miss without suggestions gets the miss hint; a miss with suggestions still gets them, and no `camel_write_file`; every miss hint is whole sentences. - camel-jbang-core, the `commands.ai` tests: 1093 run, 0 failures, 1 skipped. _Claude Code on behalf of davsclaus_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016Y4dDi5iQNgtHrDdz2NpwF -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
