Claus Ibsen created CAMEL-26301:
-----------------------------------
Summary: tokenizer EIP: an exception of the tokenizer, such as for
a null body, hangs the exchange forever
Key: CAMEL-26301
URL: https://issues.apache.org/jira/browse/CAMEL-26301
Project: Camel
Issue Type: Bug
Components: camel-langchain4j-tokenizer, camel-core
Reporter: Claus Ibsen
Fix For: 4.24.0
A message with no body sent through the tokenizer EIP
(camel-langchain4j-tokenizer) never completes: the caller waits forever.
{code:java}
from("direct:start")
.tokenize(tokenizer().byParagraph().maxTokens(1024).maxOverlap(10).using(LangChain4jTokenizerDefinition.TokenizerType.OPEN_AI).end())
.to("mock:result");
template.sendBody("direct:start", null); // never returns
{code}
*Cause*
* {{AbstractLangChain4JTokenizer.tokenize}} reads the body with
{{getIn().getBody(String.class)}}, which is null, and the splitter throws
{noformat}NullPointerException: Cannot invoke "String.split(String)" because
"text" is null{noformat}
* {{TokenizerProcessor.process(Exchange, AsyncCallback)}}
(camel-core-processor) calls {{tokenizer.tokenize(exchange)}} without a
try/catch, so the exception leaves the async process method and the callback is
never called. {{DefaultReactiveExecutor}} only logs a WARN ("Error executing
reactive work ... This exception is ignored") and the caller hangs in
{{DefaultAsyncProcessorAwaitManager.await}}.
Any exception a tokenizer throws hangs the exchange the same way, not only a
null body.
*Fix*
* {{TokenizerProcessor.process}}: catch the exception of {{tokenize}}, set it
on the exchange and call {{callback.done(true)}}, as the other EIP processors
do. Then the error handler sees a normal failure.
* The LangChain4j tokenizer could also give a clear error, or no chunks, for a
null body.
* A test with a null body (and one with a tokenizer that throws).
Found by the runtime checks of the body types of the EIPs (CAMEL-25928,
https://github.com/apache/camel/pull/28451).
_Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
