gnodet-bot commented on PR #28016: URL: https://github.com/apache/camel/pull/28016#issuecomment-6239245355
## ⚠️ Local IT results for `17aa2c99f2174b3310a3a72d782288626184453c` Mixed results against `mirror.gcr.io/ollama/ollama:0.40.6`. **Test-infra module**: `camel-test-infra-ollama` (0.40.3 → 0.40.6) **Components tested**: camel-langchain4j-chat, camel-langchain4j-agent, camel-openai | Component | Tests | Failures | Skipped | Status | |---|---|---|---|---| | camel-langchain4j-chat | 14 | 0 | 0 | ✅ PASS | | camel-langchain4j-agent | 4 (ITs) | 0 | 0 | ⚠️ timeout¹ | | camel-openai | 39 (ITs) | 1 | 21 | ❌ FAIL | **❌ Failure: `OpenAIMcpToolsIT.testMcpEchoTool`** The test asks the model to echo `"Hello from Camel"` and asserts the response contains `"hello"` or `"camel"`. With ollama 0.40.6, the model returned a meta-description instead of quoting the echoed text: ``` Response should contain echoed message from MCP echo tool but was: "The echo tool successfully returned the message you provided." ``` The MCP echo tool was correctly invoked, but the model's response no longer includes the actual echoed content — it describes the action instead. This appears to be a behavioral change in how ollama 0.40.6 formats tool call results, or a difference in how the `granite4:3b` model responds when it switches from "quoting" to "describing" the tool output. This is a **non-trivial behavioral change** requiring investigation — whether it's a model prompt-following regression or an ollama API change in result formatting. I have not pushed a fix. --- ¹ `LangChain4jAgentMcpAndCamelToolsIT` timed out (10+ min) — this is a pre-existing infrastructure issue with the complex MCP + Stdio + Ollama test combination, not related to this upgrade. <details><summary>camel-langchain4j-chat output (passing)</summary> ``` Tests run: 14, Failures: 0, Errors: 0, Skipped: 0 BUILD SUCCESS — Total time: 02:18 min ``` </details> <details><summary>camel-openai failure detail</summary> ``` [ERROR] OpenAIMcpToolsIT.testMcpEchoTool:79 Response should contain echoed message from MCP echo tool but was: The echo tool successfully returned the message you provided. ==> expected: <true> but was: <false> Tests run: 39, Failures: 1, Errors: 0, Skipped: 21 BUILD FAILURE — Total time: 11:35 min ``` </details> > _Local integration tests run by Hermes on behalf of @gnodet_ > _Tests run without `-Dci.env.name` to include ITs disabled on GitHub Actions CI_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
