[
https://issues.apache.org/jira/browse/CAMEL-25777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128628#comment-18128628
]
Claus Ibsen commented on CAMEL-25777:
-------------------------------------
Confirmed with camel run on main (4.24.0-SNAPSHOT at 58966e93ab23). The YAML
example of the tokenizer EIP page (a langChain4jParagraphTokenizer with
maxTokens 1024, maxOverlap 10 and tokenizerType OPEN_AI, run with
--dep=camel:langchain4j-tokenizer) fails at startup:
{noformat}
Error starting Camel: org.apache.camel.FailedToCreateRouteException: Failed to
create route: route1 at: >>> org.apache.camel.model.TokenizerDefinition@efe49ab
<<< in route: Route(route1)[From[direct:start] -> ...] because: jar
Caused by: java.lang.NullPointerException: jar
at
org.apache.camel.NoSuchServiceException.<init>(NoSuchServiceException.java:37)
{noformat}
So the null tokenizer name also breaks the error message: the user sees
"because: jar" instead of a missing tokenizer.
_Claude Code on behalf of davsclaus_
> camel-core - the tokenizer EIP does not work in XML and YAML routes, as only
> the Java DSL sets the tokenizer name
> -----------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25777
> URL: https://issues.apache.org/jira/browse/CAMEL-25777
> Project: Camel
> Issue Type: Bug
> Components: camel-core, camel-langchain4j-tokenizer
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> The tokenizer EIP looks like it cannot work in XML or YAML routes. Only the
> Java DSL sets the name of the tokenizer, and the reifier needs that name to
> find it.
> * {{TokenizerDefinition.tokenizerName}} is {{@XmlTransient}}. Only the
> constructor that takes an implementation sets it, and that constructor is
> what the Java DSL builder uses.
> * The XML parser ({{ModelParser.doParseTokenizerDefinition}}) creates {{new
> TokenizerDefinition()}} and calls only {{setTokenizerImplementation(...)}},
> which does not set the name. The YAML deserializer does the same.
> * {{TokenizerReifier.createProcessor}} resolves {{TOKENIZER_PATH +
> definition.tokenizerName()}}, so an XML or YAML route looks up
> {{tokenizer/null}}. That should fail with a {{NoSuchServiceException}}.
> No test runs the tokenizer EIP from XML or YAML, which is why this was not
> seen. I found it by reading the code, while adding XML and YAML tabs to the
> tokenizer EIP page (CAMEL-25774). I will add a comment when it is confirmed
> with camel run.
> A possible fix: {{tokenizerName()}} falls back to
> {{tokenizerImplementation.tokenizerName()}} when the name is not set (or
> {{setTokenizerImplementation}} sets it), plus an XML and a YAML test in
> camel-langchain4j-tokenizer.
> A smaller problem in the same model:
> {{LangChain4jTokenizerDefinition.modelName}} has {{@XmlAttribute(required =
> true)}} while its {{@Metadata}} says {{required = false}}, and the option is
> optional. The XSD therefore marks {{modelName}} as required, and a Spring XML
> route without it fails XSD validation.
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
