GitHub user GreatEugenius created a discussion: Resource Cross-Language Support
for Flink Agents
## Current Situation Analysis
Currently, in Flink Agents execution, PythonEvent can trigger PythonAction and
call Python Resource; JavaEvent can trigger JavaAction and call Java Resource.
However, Events from Python and Java APIs cannot trigger Actions from the other
language, and Actions from Python and Java APIs cannot use Resources from the
other language.
Overall, Flink Agents cross-language work can be roughly divided into three
major categories:
* Cross-language triggering between Event and Action:
PythonEvent can trigger JavaAction; JavaEvent can trigger PythonAction.
* Action cross-language Resource calls: Python Action supports calling Java
Resource; Java Action supports calling Python Resource.
* Resource cross-language Resource calls: Within Resources, we provide the
getResource method. This means that a Resource's internal implementation may
require other Resources, thus creating a need for cross-language calls.
From the user's perspective, cross-language requirements can be divided into
two phases:
1. Users build Agents using a single language: To avoid duplicate
implementations in ecosystem integration, cross-language calls between Action
and Resource need to be implemented, as well as cross-language calls within
Resources.
2. Users can mix multiple languages to build Agents: Cross-language triggering
between Event and Action needs to be implemented.
Resource implementations often depend on external SDKs, and many times external
SDKs only provide implementations in either Java or Python. It can be expected
that Resource cross-language call scenarios will gradually increase as Flink
Agents users grow and different application scenarios emerge. Therefore, we
prioritize the scenario where users build Agents using a single language,
implementing cross-language calls between Action and Resource, and
cross-language calls within Resources.
Whether in Java or Python, the cross-language support approach for Resources is
similar. Next, we will discuss how to implement Resource cross-language support
from the unified perspective of users programming in Java.
## Example: Java Calling PythonChatModel
### Create PythonChatModel
Users can register PythonChatModel by declaring ResourceDescriptor. The
acquisition and invocation of PythonChatModel is no different from other
ChatModels.
```java
@ChatModelConnection
public static ResourceDescriptor reviewAnalysisModelConnection() {
return
ResourceDescriptor.Builder.newBuilder(PythonChatModelConnection.class.getName())
.addInitialArgument("module",
"flink_agents.integrations.chat_models.tongyi_chat_model")
.addInitialArgument("clazz", "TongyiChatModelConnection")
.build();
}
@ChatModelSetup
public static ResourceDescriptor reviewAnalysisModel() {
return
ResourceDescriptor.Builder.newBuilder(PythonChatModelSetup.class.getName())
.addInitialArgument("connection", "tongyiChatModelConnection")
.addInitialArgument("module",
"flink_agents.integrations.chat_models.tongyi_chat_model")
.addInitialArgument("clazz", "TongyiChatModelSetup")
.addInitialArgument("model", "qwen-plus")
.addInitialArgument("prompt", "reviewAnalysisPrompt")
.addInitialArgument("tools",
Collections.singletonList("notifyShippingManager"))
.addInitialArgument("extract_reasoning", "true")
.build();
}
```
### GetResource
Users can obtain `PythonChatModel` through the `getResource` method:
```java
public class ChatModelAction {
public static void chat(UUID initialRequestId, String model,
List<ChatMessage> messages, RunnerContext ctx) throws Exception {
BaseChatModelSetup chatModel = (BaseChatModelSetup)
ctx.getResource(model, ResourceType.CHAT_MODEL);
}
}
```
## Implementation
### PythonResourceWrapper
To enable seamless Resource cross-language support for users, we need to
implement Wrapper classes for each ResourceType.
In Java, to implement PythonChatModelConnection and PythonChatModelSetup, we
abstracted a unified PythonResourceWrapper interface to handle these Wrapper
classes:
```java
/**
* Interface for wrapping and providing access to a Python resource.
*
* <p>This interface serves as a common abstraction for accessing Python
resources from Java,
* enabling seamless cross-language interaction and unified resource
management.</p>
*
* <p>Implementations of this interface should encapsulate the underlying
Python object or
* reference, and provide it through the {@link #getPythonResource()}
method.</p>
*/
public interface PythonResourceWrapper {
/**
* Retrieves the underlying Python resource object.
*
* @return the native Python resource wrapped by this instance
*/
Object getPythonResource();
}
```
In Python, we need to implement Wrapper classes for JavaResource:
```python
class JavaResourceWrapper(ABC):
"""Abstract base class that provides access to a Java resource.
This class defines a common interface for wrapping and accessing Java
resources,
enabling cross-language interaction in a unified manner.
"""
@abstractmethod
def get_java_resource(self) -> Any:
"""Retrieve the underlying Java resource object.
Returns:
Any: A reference to the native Java resource managed by this
wrapper.
"""
```
To ensure API consistency between Java and Python, we will add end-to-end (e2e)
tests with each code change to promptly detect API modifications and make
adaptations.
### Resource Acquisition Situation Analysis
Without considering cross-language scenarios, Resources can be directly
obtained in Action/Resource by calling the getResource method. When supporting
Resource cross-language calls, we hope that whether obtaining Resources in
Action or in Resource, Action and Resource require no changes.
More directly, the implementation form of the Resource obtained by getResource
is only related to the implementation of the current getResource method caller.
In different languages, the situations for obtaining Resources implemented in
different languages are as follows:
## Work Schedule
1. ChatModel cross-language calls
1. Support Java using PythonChatModel
1. ChatMessage: Implement mutual conversion between Java ChatMessage
and Python ChatMessage.
2. Tool: Implement PythonChatModel calling JavaTool (Metadata, Tool
Call)
3. Prompt: PythonChatModel calling JavaPrompt
2. Support Python using JavaChatModel (mainly to verify feasibility)
2. Java using other Python resources
1. Support Java using PythonEmbeddingModels
2. Support Java using PythonVectorStore
3. Support Java using PythonMCP
3. Python using other Java resources
1. Support Python using JavaEmbeddingModels
2. Support Python using JavaVectorStore
3. Support Python using JavaMCP
GitHub link: https://github.com/apache/flink-agents/discussions/338
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
