GitHub user pltbkd added a comment to the discussion: Parallel Tool Call Execution
Hi all, The parallel tool execution is indeed a highly valuable feature. Thanks for the great discussion, and sorry for jumping in so late. I've recently been working on the subagent framework and main-thread coordination (specifically aiming to achieve continuation-like execution with JDK < 21). The subagent framework introduces a similar mechanism to execute subagents in parallel, and the design proposed here is even more comprehensive. I'll bring more details about these two aspects to the community soon. I believe executeAllAsync will be a very useful addition. In the meantime, some alternative ideas came to mind: 1. What if we fanned out tool call requests into multiple independent events, executed them via multiple asynchronous actions, and finally fanned the results back in before returning them to the model? I think this might be achievable mostly with the existing event/action model, while reusing the current action state management and recovery semantics. Also, the fan-out/fan-in pattern may be useful elsewhere too. 2. Batch execution should implicitly say the executions are independent, so the execution order does not necessarily need to be respected. In fact, if order needs to be respected, I'm a bit worried that introducing pending slots doesn't help much: the recovered state of a later call cannot be used until previous calls are done. Instead, we can simply cache the later results and update the state only when all previous calls are done. 3. Moreover, if the order doesn't need to be respected, maybe we can use only one slot for the batch request, update a request-index-to-result mapping for each result, and read the corresponding result during recovery, so that partial results can still be used and the mechanism can be simpler. WDYT? GitHub link: https://github.com/apache/flink-agents/discussions/855#discussioncomment-17585625 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
