daierenao commented on code in PR #1261:
URL: https://github.com/apache/flink-agents/pull/1261#discussion_r4284018719
##########
docs/content/docs/development/calling-llms/tool_use.md:
##########
@@ -67,7 +84,7 @@ class ReviewAnalysisAgent(Agent):
"""ChatModel which focus on review analysis."""
return ResourceDescriptor(
clazz=ResourceName.ChatModel.OLLAMA_SETUP,
- ...,
+ connection="ollamaChatModelConnection",
tools=["notify_shipping_manager"], # reference the tool by its name
Review Comment:
Added `model="qwen3:8b"` to all four setups in bdf752cf and verified actual
Java/Python setup construction. Thanks!
##########
docs/content/docs/development/calling-llms/tool_use.md:
##########
@@ -311,61 +330,106 @@ public static String lookupOrder(
{{< /tabs >}}
-The supported sources are `config`, `sensory_memory`, and `short_term_memory`.
-If `source` is omitted, it defaults to `sensory_memory`.
-For Java annotation-based tools, `@ToolParam(injected = true)` is what marks
the
-parameter as framework-injected and hidden from the model; `source` and `key`
only
-configure where the value is read from.
-If the same injected parameter is declared both on the tool function and by a
-descriptor such as YAML, the declarations must resolve to the same source and
key;
-conflicting declarations fail during agent plan construction.
+Choose the source according to where the value is available when the tool runs:
-Injected parameters are part of the tool execution contract, not the model
contract:
+| Source | Runtime value | Python declaration |
+| --- | --- | --- |
+| `config` | A value in the agent execution environment configuration |
`InjectedArg.from_config("tenant_id")` |
+| `sensory_memory` | A value at a path in the current run's sensory memory |
`InjectedArg.from_sensory_memory("request.trace_id")` |
+| `short_term_memory` | A value at a path in the agent's short-term memory |
`InjectedArg.from_short_term_memory("session.tenant_id")` |
-- They are not included in the JSON schema sent to the model.
-- They are not written back to the original `ToolRequestEvent`.
-- If a model supplies an argument with the same name, the framework-provided
value takes precedence.
+For memory sources, write the value to the selected path before requesting the
+tool call. The path must exist and refer to a value, rather than a nested
memory
+object. The injection key identifies the configuration entry or memory path;
+it can differ from the tool parameter name.
+
+In Java annotations, set `@ToolParam(injected = true)` to hide the parameter
from
+the model, and use `source` and `key` to bind its value. The default source is
+`sensory_memory`; an omitted key uses the parameter name. For YAML
declarations,
+see the [`injected_args` reference]({{< ref
"docs/development/create-agent/yaml#resources" >}}),
+which covers the configuration shape, defaults, and examples.
+
+### Overrides and failures
+
+The built-in `tool_call_action` resolves injection before invoking a function
tool:
+
+- Injected parameters are excluded from the model-facing JSON schema.
+- A framework value overrides any model-supplied value for the same parameter.
+ The original `ToolRequestEvent` arguments remain unchanged.
+- A missing configuration entry, missing memory path, or invalid injected value
+ fails the tool call without invoking the function. Model-supplied values are
+ never used as a fallback.
+- If the function and a descriptor such as YAML both declare an injected
+ parameter, their source and key must agree. A conflict fails during agent
plan
+ construction, before the tool can run.
+
+This resolution is part of the built-in tool-call flow. When calling a function
+tool directly, supply its required arguments yourself.
## Parallel tool-call batches
-When `tool-call.async` is enabled, the built-in `tool_call_action` runs all
tool calls from one
-`ToolRequestEvent` as a single durable batch. Set `async.batch.parallelism` to
control concurrency:
+A model may request several tool calls in one `ToolRequestEvent`. The built-in
+`tool_call_action` can execute ordinary function or MCP tool calls as a
parallel
+batch. Use the following settings on the agent execution environment:
-- `1` — serial execution (one tool at a time).
-- `> 1` — parallel batch with a sliding window of at most that many in-flight
tool calls.
+| Option | Default | Effect on tool execution |
+| --- | --- | --- |
+| `tool-call.async` | `true` | Enables durable async tool calls. With `false`,
tools execute synchronously, one at a time. |
+| `async.batch.parallelism` | Available CPU count | Maximum concurrent calls
in one async batch. `1` executes tools one at a time while keeping async
execution enabled. |
-Java and Python support concurrent tool-call batches on all supported JDK
versions.
+For example, allow up to four concurrent calls per batch:
-Tool batches share a `num-async-threads` pool **per operator subtask** (every
key routed to
-that subtask). On JDK 21+ and in Python, chat and RAG calls also use this
pool. On older JDKs,
-Java batches use a separate pool from the Action workers. The default batch
parallelism is
-the host CPU count.
+{{< tabs "Configure Tool Parallelism" >}}
+{{< tab "Python" >}}
+```python
+from flink_agents.api.core_options import AgentExecutionOptions
-Parallel tool calls share execution capacity with other external calls. A
large tool
-batch may delay other model calls or retrieval requests. Use
`async.batch.parallelism`
-to limit how many tools run concurrently.
+agents_env.get_config().set(AgentExecutionOptions.TOOL_CALL_ASYNC, True)
+agents_env.get_config().set(AgentExecutionOptions.ASYNC_BATCH_PARALLELISM, 4)
+```
+{{< /tab >}}
+{{< tab "Java" >}}
+```java
+import org.apache.flink.agents.api.agents.AgentExecutionOptions;
-{{< hint warning >}}
-**Cross-key impact:** with defaults (`num-async-threads = 2× cores`,
`async.batch.parallelism =
-cores`), one full tool batch can use up to half the subtask pool; several hot
keys can saturate it.
-If your job mixes heavy tool batches with chat/RAG on the same subtask, lower
-`async.batch.parallelism` or increase `num-async-threads`.
-{{< /hint >}}
+agentsEnv.getConfig().set(AgentExecutionOptions.TOOL_CALL_ASYNC, true);
+agentsEnv.getConfig().set(AgentExecutionOptions.ASYNC_BATCH_PARALLELISM, 4);
+```
+{{< /tab >}}
+{{< /tabs >}}
-`async.batch.timeout.ms` applies to the whole batch. On timeout, completed
slots keep their
-outcome; slots that started but did not finish are recorded as failures; slots
that never started
-executing (for example, queued in a saturated pool) stay pending, so they are
re-executed after
-recovery instead of recording a false failure. Timeout cancellation is
best-effort; side-effecting
-tools should be idempotent or provide a reconciler.
+With async execution enabled and `async.batch.parallelism` greater than `1`,
+multiple tool calls can run concurrently. This limit also applies to batches
+created with `ctx.gather(...)`. The `num-async-threads` setting further limits
+available concurrency. Tool batches share execution capacity with other
external
+calls, so a large batch may delay other model calls or retrieval requests.
Lower
+the batch limit to leave capacity for other calls or avoid overwhelming an
+external service. Actual concurrency can be lower than the batch limit. Java
and
+Python support concurrent tool-call batches on all supported JDK versions.
{{< hint warning >}}
-**Thread reclamation:** a timeout unblocks the batch but cannot interrupt a
tool that is still
-running. Its worker thread stays in the shared `num-async-threads` pool until
the tool returns on
-its own, so a tool that never returns permanently reduces pool capacity. Bound
blocking work inside
-the tool (for example an HTTP client read timeout) rather than relying on this
batch timeout to free
-the thread.
+Calls in a parallel batch must be independent: a tool must not wait for another
+call in the same batch to complete. Concurrency is an optimization and does not
+guarantee that calls overlap or finish in request order. If one operation
depends
+on another's result, request it in a later step instead of the same batch.
{{< /hint >}}
+### Timeouts and external effects
+
+`async.batch.timeout.ms` sets an overall timeout when ordinary tool calls
execute
+as a parallel batch. Its default is `-1`; non-positive values disable it. It
does
+not set a timeout for the synchronous or sequential async tool-call paths, so
+configure timeouts for blocking operations inside each tool, such as an HTTP
+client read timeout.
Review Comment:
Clarified timeout outcomes and bypass cases in bdf752cf. The bypass wording
is scoped to ordinary-tool requests to account for mixed subagent batches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
