davsclaus opened a new pull request, #29136: URL: https://github.com/apache/camel/pull/29136
https://issues.apache.org/jira/browse/CAMEL-26564 The AI tools checked a Citrus test (`test/<name>.citrus.it.yaml`) for its YAML syntax only (CAMEL-26493). In the local-model benchmark (example test-citrus-orders), the model wrote Citrus YAML of its own in 14 of 14 runs: an `http` action with `uri`, `method` and `expectStatus`, and `log` and `assert-log` actions that Citrus does not have. `camel_write_file` wrote it, and `camel test run` then failed with "Unable to find property 'http' on class: org.citrusframework.yaml.TestActions". **The check** - `camel_write_file`, `camel_edit_file` and `camel_validate_source` check a Citrus test against the JSON schema of a test from the Citrus catalog (`org.citrusframework:citrus-catalog-schema`), for the Citrus version the CLI is built with (`citrus-version`, in a java template). An invalid test is refused, as an invalid route is. - Each item of an action list must be one Citrus test action, also inside containers such as `sequential`, `parallel` or `catch`. The schema does not check this (citrusframework/citrus#1918). - Each problem names its line. Three answers for tests the model wrote: ``` Line 15: uri, method, requestFile, contentType and expectStatus are not keys of the http action, which takes actor, client, description, receiveRequest, receiveResponse, sendRequest, sendResponse, server Line 22: log is not a Citrus test action: echo: {message: "..."} writes a message to the test log, and camel: {cli: {verify: {integration: "<name>", logMessage: "..."}}} checks that the integration logged a line Line 43: http.sendRequest.POST.body: string found, object expected; body takes data, resource, script ``` - A refused test points to the sample tests of the examples (`camel_catalog_examples` with filter `citrus`). - A number where Citrus takes text (`status: 201`) is accepted, as Citrus converts it. **Where the schema comes from** - The classpath first: the test plugin and the MCP server now have the catalog jar (274 KB), so the MCP server checks a test offline. - Else the local Maven repository, or it is downloaded once. - Without the schema, the test is checked for its YAML syntax only, and the answer says so. A test is never refused because the schema is missing. **Other changes** - `YamlValidator.filterOneOfNoise` is public, so the check uses it for the errors of a oneOf. - The MCP server page says how a Citrus test is checked. **Tests** - camel-jbang-core: the AI tool tests pass (904, with 1 skipped as before), including the new `CitrusTestChecksTest` (45) and `AuthoringToolsCitrusTest` (10). The whole module: 2569, and 294 in the serial run. - The 3 reference tests of the benchmark and all Citrus tests in camel-jbang are valid. Tests the model wrote are refused at the right lines (fixtures in `src/test/resources/citrus-orders`). - camel-jbang-plugin-test: 6. camel-yaml-dsl-validator: 345. - The MCP server, built with this change, answered `camel_validate_source` for a model test and a reference test with no download. Related: CAMEL-26563 (look up Citrus test actions from the Citrus catalog in the MCP server). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016Y4dDi5iQNgtHrDdz2NpwF -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
