[
https://issues.apache.org/jira/browse/CAMEL-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani updated CAMEL-23943:
-------------------------------------
Description:
The tool-calling loop in {{LangChain4jToolsProducer.toolsChat}} has three
related defects (all with attached failing reproducer tests):
# *Unbounded loop*: the loop is a {{do/while(true)}} with no iteration cap
({{LangChain4jToolsProducer.java:134-165}}). A model that keeps answering with
tool calls makes the producer loop forever. Reproducer
{{LangChain4jToolUnboundedLoopTest}} (mock model that always returns a tool
call) performed *3025 LLM round trips and 3025 tool executions in 15 seconds*
before the test timed out. LangChain4j itself caps this via
{{maxToolCallingRoundTrips}} / {{maxSequentialToolsInvocations}}; the component
reimplements the loop without an equivalent guard.
# *Hallucinated tool name crashes the exchange*: the tool is resolved with
{{.filter(...).findFirst().get()}} ({{LangChain4jToolsProducer.java:216-217}});
when the LLM hallucinates a tool name the exchange dies with
{{java.util.NoSuchElementException: No value present}} instead of sending a
corrective {{ToolExecutionResultMessage}} back to the model. Reproducer:
{{LangChain4jToolHallucinatedToolNameTest}}.
# *Tool exceptions swallowed and garbage fed to the LLM*: {{invokeTools}}
catches route exceptions into {{toolExchange.setException(e)}} (with a literal
{{// How to handle this exception?}} comment, lines 267-270), then sends
{{toolExchange.getIn().getBody(String.class)}} (not an error message) back to
the LLM as the tool result, and {{ExchangeHelper.copyResults}} leaks the
swallowed exception onto the main exchange. Reproducer:
{{LangChain4jToolExecutionErrorPropagationTest}}.
*Suggested fix*: add a {{maxIterations}} endpoint option (behavior change,
needs an upgrade-guide entry), return a corrective tool-result message for
unknown tool names, and feed tool failures to the LLM as an explicit error
string without failing the main exchange.
Related: CAMEL-23928 (the agent component gained these exact knobs from
AiServices; this component reimplements the loop manually).
_This issue was drafted by Claude Code on behalf of Federico Mariani._
was:
The tool-calling loop in {{LangChain4jToolsProducer.toolsChat}} has three
related defects (all with attached failing reproducer tests):
# *Unbounded loop*: the loop is {{do { ... } while (true)}} with no iteration
cap ({{LangChain4jToolsProducer.java:134-165}}). A model that keeps answering
with tool calls makes the producer loop forever. Reproducer
{{LangChain4jToolUnboundedLoopTest}} (mock model that always returns a tool
call) performed *3025 LLM round trips and 3025 tool executions in 15 seconds*
before the test timed out. LangChain4j itself caps this via
{{maxToolCallingRoundTrips}} / {{maxSequentialToolsInvocations}}; the component
reimplements the loop without an equivalent guard.
# *Hallucinated tool name crashes the exchange*: the tool is resolved with
{{.filter(...).findFirst().get()}} ({{LangChain4jToolsProducer.java:216-217}});
when the LLM hallucinates a tool name the exchange dies with
{{java.util.NoSuchElementException: No value present}} instead of sending a
corrective {{ToolExecutionResultMessage}} back to the model. Reproducer:
{{LangChain4jToolHallucinatedToolNameTest}}.
# *Tool exceptions swallowed and garbage fed to the LLM*: {{invokeTools}}
catches route exceptions into {{toolExchange.setException(e)}} (with a literal
{{// How to handle this exception?}} comment, lines 267-270), then sends
{{toolExchange.getIn().getBody(String.class)}} (not an error message) back to
the LLM as the tool result, and {{ExchangeHelper.copyResults}} leaks the
swallowed exception onto the main exchange. Reproducer:
{{LangChain4jToolExecutionErrorPropagationTest}}.
*Suggested fix*: add a {{maxIterations}} endpoint option (behavior change,
needs an upgrade-guide entry), return a corrective tool-result message for
unknown tool names, and feed tool failures to the LLM as an explicit error
string without failing the main exchange.
Related: CAMEL-23928 (the agent component gained these exact knobs from
AiServices; this component reimplements the loop manually).
_This issue was drafted by Claude Code on behalf of Federico Mariani._
> camel-langchain4j-tools: unbounded tool-calling loop, crash on hallucinated
> tool names, tool errors swallowed
> -------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-23943
> URL: https://issues.apache.org/jira/browse/CAMEL-23943
> Project: Camel
> Issue Type: Bug
> Components: camel-langchain4j-tools
> Affects Versions: 4.21.0
> Reporter: Federico Mariani
> Priority: Major
> Attachments: LangChain4jToolExecutionErrorPropagationTest.java,
> LangChain4jToolHallucinatedToolNameTest.java,
> LangChain4jToolUnboundedLoopTest.java
>
>
> The tool-calling loop in {{LangChain4jToolsProducer.toolsChat}} has three
> related defects (all with attached failing reproducer tests):
> # *Unbounded loop*: the loop is a {{do/while(true)}} with no iteration cap
> ({{LangChain4jToolsProducer.java:134-165}}). A model that keeps answering
> with tool calls makes the producer loop forever. Reproducer
> {{LangChain4jToolUnboundedLoopTest}} (mock model that always returns a tool
> call) performed *3025 LLM round trips and 3025 tool executions in 15 seconds*
> before the test timed out. LangChain4j itself caps this via
> {{maxToolCallingRoundTrips}} / {{maxSequentialToolsInvocations}}; the
> component reimplements the loop without an equivalent guard.
> # *Hallucinated tool name crashes the exchange*: the tool is resolved with
> {{.filter(...).findFirst().get()}}
> ({{LangChain4jToolsProducer.java:216-217}}); when the LLM hallucinates a tool
> name the exchange dies with {{java.util.NoSuchElementException: No value
> present}} instead of sending a corrective {{ToolExecutionResultMessage}} back
> to the model. Reproducer: {{LangChain4jToolHallucinatedToolNameTest}}.
> # *Tool exceptions swallowed and garbage fed to the LLM*: {{invokeTools}}
> catches route exceptions into {{toolExchange.setException(e)}} (with a
> literal {{// How to handle this exception?}} comment, lines 267-270), then
> sends {{toolExchange.getIn().getBody(String.class)}} (not an error message)
> back to the LLM as the tool result, and {{ExchangeHelper.copyResults}} leaks
> the swallowed exception onto the main exchange. Reproducer:
> {{LangChain4jToolExecutionErrorPropagationTest}}.
> *Suggested fix*: add a {{maxIterations}} endpoint option (behavior change,
> needs an upgrade-guide entry), return a corrective tool-result message for
> unknown tool names, and feed tool failures to the LLM as an explicit error
> string without failing the main exchange.
> Related: CAMEL-23928 (the agent component gained these exact knobs from
> AiServices; this component reimplements the loop manually).
> _This issue was drafted by Claude Code on behalf of Federico Mariani._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
