davsclaus opened a new pull request, #29035: URL: https://github.com/apache/camel/pull/29035
Fixes [CAMEL-26207](https://issues.apache.org/jira/browse/CAMEL-26207). All four bugs of the issue were real. Each is fixed in `SpringAiChatProducer`. ### 1. outputFormat BEAN, MAP or LIST dropped the question **What was wrong:** `processStructuredOutputRequest` called `request.user(u -> u.text(format))`. In Spring AI that replaces the user text, so the model got only the format instructions and not the question. With `CHAT_MULTIPLE_MESSAGES` or `CHAT_SINGLE_MESSAGE_WITH_PROMPT`, the instructions became an extra user message after the conversation. **What changed:** the instructions are now passed as the `ChatClientAttributes.OUTPUT_FORMAT` advisor parameter. Spring AI's `ChatModelCallAdvisor` then appends them to the last user message. This is what `.entity(...)` / `responseEntity(...)` do. User media and metadata are kept, and RAG and memory advisors still see the question without the instructions. ### 2. A media body without a user message failed inside Spring AI **What was wrong:** for `byte[]`, `WrappedFile` and `List<WrappedFile>`, `getUserMessageText` fell back to `""`. Spring AI rejects that with "text cannot be null or empty". Spring AI also drops media when there is no user text, so a media-only request cannot be sent through `ChatClient` at all. **What changed:** `getUserMessageText` now throws an `IllegalArgumentException` that names the `userMessage` option and the `CamelSpringAiChatUserMessage` header. It runs before any file or stream is read. A blank header now falls back to the option. This is the same choice camel-openai made ("User message configuration must be set when using an image body"). ### 3. A java.io.File, Path or InputStream image body was sent as text **What was wrong:** these bodies are not `WrappedFile` or `byte[]`. They reached the last branch and were converted with `getBody(String.class)`, so the image bytes were sent as the prompt. **What changed:** - A `File` or `Path` is sent as media when its media type is an image, audio, video or document (PDF, Word, Excel). The type comes from the `CamelSpringAiChatMediaType` header, then `CamelFileContentType`, then the file extension. It uses the same extension table and the same `maxFileSize` check as `WrappedFile`. - An `InputStream` is sent as media when `CamelSpringAiChatMediaType`, `CamelFileContentType` or `Content-Type`, or the extension in `CamelFileName`, gives such a type. It is read up to `maxFileSize` and fails if it is bigger. - A text file, a file with an unknown extension, and a stream with a text, `application/octet-stream`, `multipart/*` or missing type are still sent as text. So a text prompt from an HTTP stream body does not change. ### 4. CHAT_MULTIPLE_MESSAGES with a List of Strings failed with ClassCastException **What changed:** each element of the list is checked. A `Message` is sent as it is. A `String` (any `CharSequence`) is sent as a `UserMessage`. Any other element fails with an `IllegalArgumentException` that names its type and the expected types. ### Tests New `SpringAiChatBodyHandlingTest` (16 tests) uses a fake `ChatModel` that records the `Prompt`. No external service is needed. Run against the unchanged main code, 12 of the tests fail. The other 4 guard behaviour that stays the same: `byte[]` with a user message, a text file, and a text or octet-stream stream. The module's unit tests pass (36). The ITs ran against a local Ollama, and `SpringAiChatStructuredOutputIT` (10) passes with the change. `SpringAiChatMultimodalIT.testMultimodalWithUserMessageObject`, `SpringAiChatVectorMemoryIT` and `SpringAiChatVectorStoreIT` fail the same way on main. The causes are a local model without multimodal support and a missing `embeddinggemma` model, so these failures are not caused by this change. ### Docs and generated files - `@BodyTypes` text of the endpoint updated. The old text said a stream or `java.io.File` "is sent as its text". The component JSON was regenerated and copied to camel-catalog. The carriers are unchanged, and no yaml-dsl-validator test references `spring-ai-chat`. - Component doc: new input types, the user-message requirement for media, `String` elements in a conversation, and where the format instructions go. - Upgrade guide 4.24: new `camel-spring-ai-chat` section, placed before `camel-spring-rabbitmq`. ### Not changed - A `byte[]` body still defaults to `image/png` and reads only the `CamelSpringAiChatMediaType` header. It does not read `Content-Type`. - A `WrappedFile` of a `.txt` file is still sent as `text/plain` media, as before. - A `UserMessage` body with media and empty text is passed to Spring AI as it is. _Claude Code on behalf of Claus Ibsen (davsclaus)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
