Claus Ibsen created CAMEL-26069:
-----------------------------------
Summary: camel-aws-bedrock - the invoke operations send the body
as its toString(), and the image and embeddings streaming operations give an
empty text
Key: CAMEL-26069
URL: https://issues.apache.org/jira/browse/CAMEL-26069
Project: Camel
Issue Type: Bug
Components: camel-aws2
Affects Versions: 4.18.0, 4.22.0, 4.23.0
Reporter: Claus Ibsen
Fix For: 4.24.0
Found while adding the catalog body types of aws-bedrock (CAMEL-24961).
h4. 1. The invoke operations send the body as its toString()
invokeTextModel, invokeImageModel, invokeEmbeddingsModel and the three
streaming operations build the request of the model with
{{SdkBytes.fromUtf8String(String.valueOf(exchange.getMessage().getBody()))}}
({{BedrockProducer}}, main ed3d719b379c, lines 170, 217, 269 and 567), so no
type conversion is done, and only a String body works. Checked on
4.24.0-SNAPSHOT with {{camel run}} against a fake of the AWS API that logs the
requests it gets, with the content type headers set:
* a byte[] of the JSON request is sent as {{[B@4d774bb6}};
* an InputStream is sent as
{{org.apache.camel.converter.stream.ByteArrayInputStreamCache@620106cb}};
* a java.io.File is sent as its path (and a GenericFile from a file consumer as
{{GenericFile[...]}}, read in the code);
* a Map is sent as {{\{inputText=Hello\}}};
* no body is sent as the text {{null}}.
*Proposed change*: read the body with {{getMandatoryBody(String.class)}}, or as
bytes with {{SdkBytes.fromByteArray}}.
h4. 2. invokeImageModelStreaming and invokeEmbeddingsModelStreaming give an
empty text
Both call {{invokeTextModelStreaming}} (lines 574-584), whose stream parsers
read the generated text of each chunk ({{outputText}} for Titan). A chunk of an
image or of an embedding has no such text. Checked on 4.24.0-SNAPSHOT with
{{camel run}} against a fake of the AWS API that logs the requests it gets,
streaming one chunk with {{images}} for {{amazon.titan-image-generator-v1}} and
one with {{embedding}} for {{amazon.titan-embed-text-v1}}: the body is an empty
String, with {{CamelAwsBedrockChunkCount}} 1.
*Proposed change*: parse the image and embedding chunks, or remove these two
operations if the models do not stream.
h4. 3. pojoRequest=true without the modelId option fails after the call
With pojoRequest=true the body is an {{InvokeModelRequest}} that carries its
model id, but the response is read with the model of the endpoint: {{switch
(modelId)}} at line 292 fails with {{NullPointerException: Cannot invoke
"String.hashCode()"}} after the model was called. Checked on 4.24.0-SNAPSHOT
with {{camel run}} against a fake of the AWS API that logs the requests it
gets. The streaming operations fail with "Model ID cannot be null"
({{StreamParserFactory}}, lines 35-37), although the request carries the model
(read in the code).
*Proposed change*: take the model id of the request in POJO mode.
h4. 4. amazon.titan-embed-text-v2:0 is accepted by invokeTextModel
The docs say that an embedding model fails fast with invokeTextModel, and
{{amazon.titan-embed-text-v1}} does, but {{amazon.titan-embed-text-v2:0}} is
listed with the Titan text models (line 297), so invokeTextModel gives the JSON
of the embedding as text. Checked on 4.24.0-SNAPSHOT with {{camel run}} against
a fake of the AWS API that logs the requests it gets.
h4. 5. No operation fails with a NullPointerException
The operation option has no default, and with no CamelAwsBedrockOperation
header, {{switch (determineOperation(exchange))}} at line 80 fails with
{{NullPointerException: Cannot invoke "...BedrockOperations.ordinal()" because
the return value of "...determineOperation(...)" is null}}. Checked on
4.24.0-SNAPSHOT with {{camel run}} against the AWS emulator of {{camel infra
run aws}}.
*Proposed change*: fail with a message that says to set the operation option or
header.
h4. Also read in the code, not run
* The streaming operations do not resolve an inference profile id or an ARN to
the model ({{StreamParserFactory.getParser}} gets the raw model id, lines 610
and 626), so {{us.anthropic.claude-3-haiku-20240307-v1:0}} fails with
"Unsupported model for streaming". CAMEL-24224 fixed this for invokeTextModel
only.
* converseStream never sets the {{CamelAwsBedrockGuardrailTrace}} header:
{{ConverseStreamHandler.setGuardrailTrace}} is never called, so lines 914-916
never run. It does not set {{CamelAwsBedrockConverseOutputMessage}} either,
which the docs list.
* Header types in the catalog: {{CamelAwsBedrockGuardrailOutput}} is a
{{List<GuardrailOutputContent>}}, not a GuardrailAssessment;
{{CamelAwsBedrockGuardrailTrace}} is a {{GuardrailTraceAssessment}}, not the
enum GuardrailTrace; converseStream reads {{CamelAwsBedrockGuardrailConfig}} as
a {{GuardrailStreamConfiguration}}, so the GuardrailConfiguration the docs name
is ignored there.
* From the documented response formats, not run: AI21 Jamba gives a null body
with invokeTextModel (line 509 reads {{completions}}, Jamba answers with
{{choices}}), and the streaming of Jamba and Nova gives an empty text, as their
chunks do not match the Titan and Claude parsers ({{StreamParserFactory}},
lines 65-72).
_Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
