davsclaus opened a new pull request, #28892: URL: https://github.com/apache/camel/pull/28892
Fixes [CAMEL-26069](https://issues.apache.org/jira/browse/CAMEL-26069). ## 1. The invoke operations sent the body as its toString() **Was:** `invokeTextModel`, `invokeImageModel`, `invokeEmbeddingsModel` and the streaming invoke operations built the request with `SdkBytes.fromUtf8String(String.valueOf(body))`. A byte[] went out as `[B@...`, a stream or a file as its class name or path, a Map as `{inputText=Hello}`, and no body as `null`. **Now:** the body is converted with `getMandatoryBody(byte[].class)`, so String, byte[], InputStream and File (also a GenericFile) are sent as their content. A `Map` or a Jackson `JsonNode` is marshalled to JSON (the request of these models is JSON, and the component already uses Jackson). No body fails with `InvalidPayloadException`. The catalog body types (`producerIn`) are updated to match. ## 2. invokeImageModelStreaming and invokeEmbeddingsModelStreaming gave an empty text **Was:** both call `invokeTextModelStreaming`, whose parsers read the text of each chunk. An image or embedding chunk has no text, so the body was "". **Now:** the AWS model cards list only the Invoke API for the image and embedding models; they do not stream. All three streaming operations now fail with an `IllegalArgumentException` for an image, embedding, rerank or video/speech model, and the message points to `invokeImageModel` / `invokeEmbeddingsModel`. For a text model the two operations still work as `invokeTextModelStreaming`, so nobody who uses them with a text model breaks. ## 3. pojoRequest=true without the modelId option **Was:** the response was read with the model of the endpoint: `switch (null)` gave an NPE after the model was called. Streaming failed with "Model ID cannot be null". **Now:** the response and the stream parser use the model of the request. `setResponseText(result, message)` stays (protected) and delegates to a new overload that takes the model id. ## 4. amazon.titan-embed-text-v2:0 was accepted by invokeTextModel Moved from the Titan text list to the embedding list, so it fails fast like `amazon.titan-embed-text-v1`. ## 5. No operation failed with a NullPointerException Now: `IllegalArgumentException: Operation must be specified with the operation option or the CamelAwsBedrockOperation header`. ## Also from the issue - **Streaming with an inference profile id or ARN:** the stream parser now gets the resolved foundation model (`resolveFoundationModelId`), as `invokeTextModel` does since CAMEL-24224. - **converseStream guardrail trace:** the handler now reads `trace().guardrail()` of the metadata event, so `CamelAwsBedrockGuardrailTrace` is set. `ConverseStreamHandler.StreamMetadata` holds a `GuardrailTraceAssessment` (was the enum `GuardrailTrace`, never set). With `guardrailTrace=true` on the endpoint, converseStream now also sets `trace(ENABLED)` on the request; before it only changed the stream processing mode, so no trace came back. - **converseStream and the GuardrailConfig header:** it now takes the `GuardrailConfiguration` the docs name (copied into a `GuardrailStreamConfiguration`), as well as a `GuardrailStreamConfiguration`. - **CamelAwsBedrockConverseOutputMessage:** documented as "converse only". converseStream does not build a Message from the stream (tool use blocks are not collected), so the docs were fixed rather than inventing a partial Message. - **Header types in the catalog:** `CamelAwsBedrockGuardrailOutput` is `List<GuardrailOutputContent>`, `CamelAwsBedrockGuardrailTrace` is `GuardrailTraceAssessment`. - **AI21 Jamba:** `invokeTextModel` gives `choices` when the response has no `completions` (Jurassic-2). New `Ai21StreamParser` (chat completions chunks: `choices[0].delta.content`, `finish_reason`, `usage.completion_tokens`) and `NovaStreamParser` (`contentBlockDelta.delta.text`, `messageStop.stopReason`, `metadata.usage.outputTokens`) replace the Titan and Claude parsers these models were routed to. The formats come from the AWS Nova streaming docs and the chat completions format of Jamba; not run against AWS. ## Tests - New `BedrockProducerRequestTest` (21 tests) with mocked sync and async clients; the async mock feeds chunks to the real response handler. It covers the body types, no body, no operation, POJO mode without modelId (sync and streaming), an inference profile in streaming, image/embedding streaming, Nova and Jamba streaming, the Jamba response, and the converseStream guardrail config and trace. - `BedrockInvokeTextModelUnsupportedModelTest` also covers `amazon.titan-embed-text-v2:0`; `ConverseStreamHandlerTest` covers the guardrail trace of the metadata event. - With the `src/main` changes stashed, 19 of the 21 new tests fail, as do the titan-embed-v2 case and the guardrail trace test. The 2 that pass are the String body and the JsonNode body: their `toString()` is already the JSON. - `mvn clean install -Psourcecheck` of the module: 170 tests, 0 failures. ## Docs and generated files Component doc (request body, operation, streaming of image/embedding models, Jamba, inference profiles, headers), the regenerated `aws-bedrock.json` (headers and body types) copied into the catalog with the doc, the endpoint DSL header javadoc, and an upgrade guide entry for 4.24: the body conversion, no body failing, titan-embed v2 and image/embedding streaming failing, and the header type. ## Not changed - `invokeImageModelStreaming` and `invokeEmbeddingsModelStreaming` are kept, not deprecated. **Open question:** deprecate them for removal, since they only work for text models? - On converseStream, `guardrailTrace` still also picks the stream processing mode (ASYNC when true). That looks unrelated to the trace, but changing it changes latency behaviour, so I left it. - aws-bedrock-agent-runtime is CAMEL-26071 and is not touched here. _Claude Code on behalf of Claus Ibsen (davsclaus)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
