davsclaus opened a new pull request, #29323:
URL: https://github.com/apache/camel/pull/29323

   https://issues.apache.org/jira/browse/CAMEL-26690
   
   Part 1 of 4 for CAMEL-26690, follow-ups from the r28 run of the local-model 
benchmark. The four PRs are independent of each other: each was built and 
tested alone on main.
   
   Steps 1 and 2 of the benchmark's test-citrus-orders never passed (0/7 in r26 
and r28). Every refusal was right (all 39 tests the model wrote were checked 
against the Citrus 5.1.0 schema), but the messages did not say how to write the 
test.
   
   - **The http action.** The model put `uri`, `method` and `expectStatus` 
straight under `http:`, and the refusal listed only the keys of the http 
action. It now adds that an http action of a client either sends the request 
(`sendRequest`) or receives the response (`receiveResponse`), so the request 
and the response it expects are two http actions, and gives "Write lines 14-19 
as:" with the two actions built from the model's own values. The lines are 
checked against the schema before they are offered.
   - **The body.** 5 of 7 runs had the whole case right except the body check, 
written as `text`, `contains` or `equalTo`. The message now says the text goes 
under `data:`, the body the test expects (Citrus compares a JSON body as JSON), 
shows the jsonPath `validate` for single fields, and gives the lines to write.
   - **Starting the integration.** `camel test run` runs only the test. A test 
that calls the integration but does not start it now gets the `camel: cli: run` 
action with the project's route file. No test written in r26 or r28 step 1 
started the integration.
   - The refusal no longer points to `camel_catalog_examples`, a tool the 
model's client did not offer.
   
   **Checks**
   - All 8 r28 step 1 writes and all 16 r26 step 1 and 2 writes get the hints, 
and writing the suggested lines into the model's forms removes the problem 
(unit tests).
   - No message on the 27 bundled tests, the step references and the 13 valid 
step 3 edits.
   - Tests pass: the `ai` package of camel-jbang-core (949, 1 skipped as 
before), camel-yaml-dsl-validator (350), camel-jbang-mcp (539).
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   
   https://claude.ai/code/session_016Y4dDi5iQNgtHrDdz2NpwF
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to