[
https://issues.apache.org/jira/browse/CAMEL-26301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18132723#comment-18132723
]
Claus Ibsen commented on CAMEL-26301:
-------------------------------------
Backport to camel-4.18.x merged via https://github.com/apache/camel/pull/28519.
The camel-4.22.x backport https://github.com/apache/camel/pull/28518 is still
open.
_Claude Code on behalf of davsclaus_
> tokenizer EIP: an exception of the tokenizer, such as for a null body, hangs
> the exchange forever
> -------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26301
> URL: https://issues.apache.org/jira/browse/CAMEL-26301
> Project: Camel
> Issue Type: Bug
> Components: camel-core, camel-langchain4j-tokenizer
> Reporter: Claus Ibsen
> Assignee: Guillaume Nodet
> Priority: Major
> Fix For: 4.18.6, 4.22.3, 4.24.0
>
>
> A message with no body sent through the tokenizer EIP
> (camel-langchain4j-tokenizer) never completes: the caller waits forever.
> {code:java}
> from("direct:start")
>
> .tokenize(tokenizer().byParagraph().maxTokens(1024).maxOverlap(10).using(LangChain4jTokenizerDefinition.TokenizerType.OPEN_AI).end())
> .to("mock:result");
> template.sendBody("direct:start", null); // never returns
> {code}
> *Cause*
> * {{AbstractLangChain4JTokenizer.tokenize}} reads the body with
> {{getIn().getBody(String.class)}}, which is null, and the splitter throws
> {noformat}NullPointerException: Cannot invoke "String.split(String)" because
> "text" is null{noformat}
> * {{TokenizerProcessor.process(Exchange, AsyncCallback)}}
> (camel-core-processor) calls {{tokenizer.tokenize(exchange)}} without a
> try/catch, so the exception leaves the async process method and the callback
> is never called. {{DefaultReactiveExecutor}} only logs a WARN ("Error
> executing reactive work ... This exception is ignored") and the caller hangs
> in {{DefaultAsyncProcessorAwaitManager.await}}.
> Any exception a tokenizer throws hangs the exchange the same way, not only a
> null body.
> *Fix*
> * {{TokenizerProcessor.process}}: catch the exception of {{tokenize}}, set it
> on the exchange and call {{callback.done(true)}}, as the other EIP processors
> do. Then the error handler sees a normal failure.
> * The LangChain4j tokenizer could also give a clear error, or no chunks, for
> a null body.
> * A test with a null body (and one with a tokenizer that throws).
> Found by the runtime checks of the body types of the EIPs (CAMEL-25928,
> https://github.com/apache/camel/pull/28451).
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
