luigidemasi commented on code in PR #27850: URL: https://github.com/apache/camel/pull/27850#discussion_r4276448598
########## dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/common/SemanticMediaInput.java: ########## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.camel.dsl.jbang.core.common; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Base64; +import java.util.List; +import java.util.Locale; +import java.util.Map; + +import org.apache.camel.util.json.JsonArray; +import org.apache.camel.util.json.JsonObject; + +/** Media input shared by the CLI and TUI; independent of expert implementations. */ +public final class SemanticMediaInput { + private static final String PARTS = "$semanticInput"; + + private SemanticMediaInput() { + } + + public static boolean isMedia(Object input) { + return input instanceof Map<?, ?> map && map.get(PARTS) instanceof List<?>; + } + + public static List<?> parts(Object input) { + return isMedia(input) ? (List<?>) ((Map<?, ?>) input).get(PARTS) : List.of(); + } + + /** Read only an explicitly requested local image. This must run off the TUI render thread. */ + public static Object readImage(String value) { + if (!value.startsWith("image:")) { + return value; + } + Path path = Path.of(value.substring(6)); + String name = path.getFileName().toString().toLowerCase(Locale.ROOT); + String type = name.endsWith(".png") ? "image/png" + : name.endsWith(".jpg") || name.endsWith(".jpeg") ? "image/jpeg" + : name.endsWith(".gif") ? "image/gif" : name.endsWith(".webp") ? "image/webp" : null; + if (type == null) { + throw new IllegalArgumentException("image: requires a local PNG, JPEG, GIF or WebP file"); + } + try { + String data = "data:" + type + ";base64," + Base64.getEncoder().encodeToString(Files.readAllBytes(path)); Review Comment: Addressed in [b5626da0ab5b](https://github.com/apache/camel/commit/b5626da0ab5be210adde5f250c8f6ce804bd7de5). The limit is configured on the expert bean as `maxImageBytes`, defaulting to 10,000,000 bytes per image for OpenAI. The expert advertises its configured value in runtime metadata. The CLI reads that metadata before opening an image file, and both TUI evaluation screens use the selected expert's advertised limit. The shared loader checks the file size and bounds the actual read, including when a file grows after the size check. Inline images are checked before tooling submission too. The expert enforces the same limit for route input and direct calls, with bounded route-stream caching. Regression tests cover the boundary, configured limits, file growth, missing metadata and rejection before CLI/TUI evaluation. The original 64 MiB file probe now rejects cleanly in a JVM with a 32 MiB heap. The target remains 4.24.0, and the GitHub milestone is set accordingly; no 4.23 backport is planned. _AI-generated by OpenAI Codex on behalf of @luigidemasi._ ########## components/camel-ai/camel-semantic/src/main/java/org/apache/camel/semantic/SemanticInput.java: ########## @@ -0,0 +1,152 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.semantic; + +import java.io.InputStream; +import java.util.Base64; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Set; + +import org.apache.camel.Exchange; +import org.apache.camel.NoTypeConversionAvailableException; +import org.apache.camel.StreamCache; +import org.apache.camel.semantic.SemanticExpert.InputType; + +/** + * Explicit, JSON-compatible input envelope for ordered text and image parts. The reserved {@value #PARTS} key + * distinguishes media from ordinary application state. Values are data: no file or URL is resolved by this class. + */ +public final class SemanticInput { + public static final String PARTS = "$semanticInput"; + + private SemanticInput() { + } + + public static Map<String, Object> ofParts(List<Map<String, Object>> parts) { + Map<String, Object> input = Map.of(PARTS, parts.stream().map(Map::copyOf).toList()); + validate(input, Set.of(InputType.TEXT, InputType.IMAGE)); + return input; + } + + public static Map<String, Object> text(String text) { + return Map.of("type", "text", "text", text); + } + + /** An image data URL, with an optional expert-specific detail setting supplied separately in the part. */ + public static Map<String, Object> image(String data) { + return Map.of("type", "image", "data", data); + } + + /** Binary image part with an explicit MIME type. No image interpretation or inference is performed. */ + public static Map<String, Object> image(byte[] data, String mediaType) { + return image("data:" + mediaType + ";base64," + Base64.getEncoder().encodeToString(data)); + } + + /** Normalize a self-describing image string; ordinary strings remain text. */ + public static Object normalize(Object input) { + return input instanceof String text && text.startsWith("data:image/") + ? ofParts(List.of(image(text))) : input; + } + + /** + * Select an image body using Content-Type. Streams become replayable Camel stream caches before reading, so later + * route steps can still consume the body. Headers do not affect explicitly selected headers or variables. + */ + public static Object body(Exchange exchange) throws NoTypeConversionAvailableException { + Object input = exchange.getMessage().getBody(); + String contentType = exchange.getMessage().getHeader(Exchange.CONTENT_TYPE, String.class); + String mediaType = contentType == null ? "" : contentType.split(";", 2)[0].trim().toLowerCase(Locale.ROOT); + if (mediaType.startsWith("audio/") || mediaType.startsWith("video/")) { + throw new IllegalArgumentException("Audio and video bodies are not supported by the selected input contract"); + } + if (!mediaType.startsWith("image/")) { + return normalize(input); + } + if (!mediaType.matches("image/[a-z0-9.+-]{1,100}")) { + throw new IllegalArgumentException("Invalid image Content-Type"); + } + if (input instanceof InputStream && !(input instanceof StreamCache)) { + input = exchange.getContext().getTypeConverter().mandatoryConvertTo(StreamCache.class, exchange, input); + exchange.getMessage().setBody(input); + } + if (input instanceof StreamCache cache) { + try { + cache.reset(); + byte[] data = exchange.getContext().getTypeConverter().mandatoryConvertTo(byte[].class, exchange, cache); + return ofParts(List.of(image(data, mediaType))); + } finally { + cache.reset(); + } + } + if (input instanceof byte[] data) { + return ofParts(List.of(image(data, mediaType))); + } + if (input instanceof String text) { + return text.startsWith("data:image/") + ? normalize(text) + : ofParts(List.of(image("data:" + mediaType + ";base64," + text))); Review Comment: Addressed the early size checks and documentation in [b5626da0ab5b](https://github.com/apache/camel/commit/b5626da0ab5be210adde5f250c8f6ce804bd7de5), and added regression coverage for the exact body/header path. `SemanticInput.body()` reads the message's `Exchange.CONTENT_TYPE` header and normalizes the selected body. Base64 syntax validation already runs in the OpenAI expert's `validateState` path and in the native endpoint before the API call. The new test supplies empty, whitespace, malformed and multi-megabyte plaintext String bodies with `Content-Type: image/png`; all are rejected with zero provider calls. The expert's configured `maxImageBytes` is now also checked before concatenating a base64 body into a data URL or encoding binary input. Streams are bounded while being cached. The docs explicitly require base64 for image String bodies and explain where validation happens. The existing base64 validator remains responsible for the full syntax scan. _AI-generated by OpenAI Codex on behalf of @luigidemasi._ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
