davsclaus commented on code in PR #27569:
URL: https://github.com/apache/camel/pull/27569#discussion_r4219987755
##########
components/camel-ai/camel-langchain4j-ingest/src/main/java/org/apache/camel/component/langchain4j/ingest/LangChain4jIngestProducer.java:
##########
@@ -290,15 +294,64 @@ private IngestResult ingestUnlessFiltered(Exchange
exchange, String documentId,
if (announced != null) {
mediaService.checkSize(documentId, announced);
}
- // bytes, not a String - a charset conversion would corrupt them.
A null body stays
- // EMPTY; anything else must convert, so a wrong body type (a
POJO, say) fails as an
- // InvalidPayloadException instead of being answered EMPTY by a
silent null conversion
- byte[] bytes = exchange.getMessage().getBody() == null
- ? null
- : exchange.getMessage().getMandatoryBody(byte[].class);
- return mediaService.ingest(documentId, bytes, mimeType, medium);
- }
- return textService.ingest(documentId,
exchange.getMessage().getBody(String.class));
+ return mediaService.ingest(documentId, mediaBody(exchange),
mimeType, medium);
+ }
+ return textService.ingest(documentId, textBody(exchange));
+ }
+
+ /**
+ * The media body as bytes, not a String - a charset conversion would
corrupt them. A null body stays EMPTY;
+ * anything else must convert, so a wrong body type (a POJO, say) fails as
an InvalidPayloadException instead of
+ * being answered EMPTY by a silent null conversion. With maxDocumentSize
set, a body not yet in memory is read only
+ * one byte past the cap, enough for the size check to refuse it.
+ */
+ private byte[] mediaBody(Exchange exchange) throws
InvalidPayloadException, IOException {
+ Object body = exchange.getMessage().getBody();
+ int limit = readLimit();
+ InputStream stream = body == null || body instanceof byte[] || limit < 0
+ ? null : exchange.getMessage().getBody(InputStream.class);
+ if (stream == null) {
+ return body == null ? null :
exchange.getMessage().getMandatoryBody(byte[].class);
+ }
+ try (stream) {
+ return stream.readNBytes(limit);
+ }
+ }
+
+ /**
+ * The text body. With maxDocumentSize set, a body not yet a String is
read only one character past the cap, enough
+ * for the size check to refuse it.
+ */
+ private String textBody(Exchange exchange) throws IOException {
+ Object body = exchange.getMessage().getBody();
+ int limit = readLimit();
+ Reader reader = body == null || body instanceof String || limit < 0
+ ? null : readerOf(exchange);
+ if (reader == null) {
+ return exchange.getMessage().getBody(String.class);
+ }
+ try (reader) {
+ StringBuilder text = new StringBuilder();
+ char[] chunk = new char[8192];
+ int read;
+ while (text.length() < limit
+ && (read = reader.read(chunk, 0, Math.min(chunk.length,
limit - text.length()))) != -1) {
+ text.append(chunk, 0, read);
+ }
+ return text.toString();
+ }
+ }
+
+ /** The body as a Reader, or null; a File or a Path converts to a
BufferedReader, not to a Reader. */
+ private static Reader readerOf(Exchange exchange) {
Review Comment:
Optional, not blocking: with a cap, a `File`/`Path` body is decoded through
`Files.newBufferedReader`, which uses UTF-8 when no exchange charset is set and
throws `MalformedInputException` on bad bytes. Without a cap it goes through
`IOConverter.toString(Path)`, which falls back to `new String(bytes)`: the
platform charset, with bad bytes replaced. So a Latin-1 file that ingests
without a cap fails once `maxDocumentSize` is set.
The PR description says this is intended, to match file-consumer bodies, and
that makes sense. It might be worth one sentence in the component doc too, so
users don't find it only by hitting the exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
