[
https://issues.apache.org/jira/browse/CAMEL-26207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18136911#comment-18136911
]
Claus Ibsen commented on CAMEL-26207:
-------------------------------------
PR: https://github.com/apache/camel/pull/29035
All four bugs are fixed:
# outputFormat BEAN, MAP or LIST: the format instructions are now appended to
the last user message (via the OUTPUT_FORMAT advisor context, as entity(...)
does) instead of replacing the question.
# A media body without a user message now fails early with an
IllegalArgumentException that names the userMessage option and the
CamelSpringAiChatUserMessage header (same choice as camel-openai). A default
text was not chosen.
# A java.io.File, Path or InputStream of an image, audio, video or document
(media type from the headers or the file name) is now sent as media. A text
one, or one of unknown type, is still sent as text.
# CHAT_MULTIPLE_MESSAGES takes String elements as UserMessage, and fails other
elements with a clear message.
_Claude Code on behalf of Claus Ibsen (davsclaus)_
> camel-spring-ai-chat - outputFormat BEAN, MAP or LIST drops the user's
> question, and a media body without a user message fails
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26207
> URL: https://issues.apache.org/jira/browse/CAMEL-26207
> Project: Camel
> Issue Type: Bug
> Components: camel-spring-ai
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> Found while adding the catalog body types of spring-ai-chat (CAMEL-24961, PR
> #28401).
> h4. 1. outputFormat BEAN, MAP or LIST drops the user's question
> {{SpringAiChatProducer.processStructuredOutputRequest}} (main 2a465aa4814c,
> line 980) calls {{request.user(u -> u.text(format))}}, which replaces the
> user text set earlier with the converter's format instructions. The model
> gets only "Your response should be in JSON format..." and not the question,
> so the structured answer is unrelated to the input. Checked on
> 4.24.0-SNAPSHOT with a fake {{ChatModel}}.
> Proposed: append the format instructions to the user text (as Spring AI's
> {{.entity(...)}} does) instead of replacing it.
> h4. 2. byte[], WrappedFile or List<WrappedFile> without a user message fail
> {{getUserMessageText}} (line 379) falls back to {{""}} when neither the
> {{CamelSpringAiChatUserMessage}} header nor the {{userMessage}} option is
> set; Spring AI then rejects the request with "text cannot be null or empty",
> so an image or audio body alone cannot be sent. Checked on 4.24.0-SNAPSHOT
> with a fake {{ChatModel}}.
> Proposed: use a default text (for example "Describe this content") or fail
> early with a message that names the user message header.
> h4. 3. A java.io.File or InputStream image body is sent as text
> A {{File}} or {{InputStream}} body is neither {{WrappedFile}} nor {{byte[]}},
> so it reaches the last branch (line 257) and is converted with
> {{getBody(String.class)}}: the PNG bytes are sent as the prompt text. Checked
> on 4.24.0-SNAPSHOT with a fake {{ChatModel}}.
> Proposed: handle {{File}}, {{Path}} and {{InputStream}} like {{byte[]}}
> (media with a MIME type from the headers or the file name).
> h4. 4. CHAT_MULTIPLE_MESSAGES with a List of Strings fails with
> ClassCastException
> {{processMultipleMessages}} (line 325) casts the body to {{List<Message>}}
> without checking, so a {{List<String>}} body fails with a raw
> {{ClassCastException}} inside Spring AI. Checked on 4.24.0-SNAPSHOT with a
> fake {{ChatModel}}.
> Proposed: convert String elements to {{UserMessage}}, or fail with a message
> that says a List of Message is required.
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
