Claus Ibsen created CAMEL-26250:
-----------------------------------
Summary: camel-google-vertexai - generateMultimodal silently drops
a stream or File image, and rawPredict rejects byte[], stream and File JSON
bodies
Key: CAMEL-26250
URL: https://issues.apache.org/jira/browse/CAMEL-26250
Project: Camel
Issue Type: Bug
Components: camel-google
Reporter: Claus Ibsen
Fix For: 4.24.0
Found while adding the catalog body types of google-vertexai (CAMEL-24961, PR
#28404).
h4. 1. generateMultimodal silently drops a stream, File or byte[] image
{{GoogleVertexAIProducer.generateMultimodal}} (main 2a465aa4814c, lines
315-322) uses the body only when it is a {{byte[]}} with the
{{CamelGoogleVertexAIMediaMimeType}} header set, or a String without a prompt.
An {{InputStream}}, {{File}} or {{GenericFile}} body (a file or HTTP consumer),
or a byte[] without the mime type header, is ignored without an error, so only
the prompt is sent and the model answers without the image. Checked on
4.24.0-SNAPSHOT with a JDK {{HttpServer}} fake of the Gemini API.
Proposed: convert the body to byte[] with the type converter, and fail when
there is media without a mime type (or guess it from the file name).
h4. 2. rawPredict and streamRawPredict reject byte[], stream and File bodies
{{buildRawPredictRequestBody}} (lines 491-523) accepts a String or a Map only
and throws "Request body must be a JSON String, Map, or plain text prompt" for
anything else, so JSON coming from a file or HTTP consumer fails. Checked on
4.24.0-SNAPSHOT with a mocked {{PredictionServiceClient}}.
Proposed: convert other bodies to String with the type converter before the
checks.
h4. 3. Six declared headers are never used
{{GoogleVertexAIConstants}} declares {{CamelGoogleVertexAIChatMessages}},
{{SystemInstruction}}, {{SafetySettings}} (lines 62-69), {{SafetyRatings}},
{{ContentBlocked}} (lines 85-88) and {{AnthropicVersion}} (line 151), and the
docs list them, but the producer neither reads nor sets them. Read in the code,
not run.
Proposed: implement them or remove them from the constants and the docs.
_Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
