Xiao-zhen-Liu opened a new issue, #3915: URL: https://github.com/apache/texera/issues/3915
## Overview We are in the process of merging a chat assistant service into `main`. The chat assistant is powered by a multi-agent system based on OpenAI Agents SDK, and can help users construct Texera workflows through natural language conversations. This issue tracks the design and PR plan of this project. ## Design The chat assistant service will be a new micro-service in Texera: - The backend is a Python web-server and connects to an external LLM API (currently it is OpenAI API, non-OpenAI models are also [supported](https://openai.github.io/openai-agents-python/models/litellm/) by Agents SDK) - Frontend communicates with the service through a single WebSocket session. Currently a chat session is bundled with a WebSocket session, and the chat session will be lost when the chat interface is closed. We plan to add session persistence in the future. The following diagram shows the lifecycle of a user's frontend establishing a WebSocket connection to the chat assistant service, using the connection to send user chat messages to OpenAI's service, and receiving streamed response messages from OpenAI's service.  The agents are equipped with [function-calling](https://platform.openai.com/docs/guides/function-calling) capabilities and can interact with user's interface (e.g., add an operator on the canvas). Functions are defined in the Python server and need WebSocket communications with the frontend. The following diagram shows the lifecycle of a function call.  ### Relationship with Existing AI Components There are existing lightweight AI components in the current Texera codebase. As these components are written in TexeraWebApplication (Scala), the new service will not reuse code from these components. The following diagram shows their relationships. <img width="1810" height="747" alt="Image" src="https://github.com/user-attachments/assets/bb9b88f7-a148-4b92-acba-1f0093307724"; /> ### Python ChatAssistantService Architecture The following diagram shows the architecture of the Python ChatAssistantService server. <img width="1405" height="914" alt="Image" src="https://github.com/user-attachments/assets/ad6b3f0f-258b-4b51-b547-64c2836d2236"; /> ## PR Plan The PRs for this project will be created and reviewed incrementally. Each PR is designed to be testable. #### PR1: Add basic Python server skeleton - Add a basic Python web server with only a health-check endpoint. - Set up project files and CI - How to test: Verify if web-server & health-check works. #### PR2: Add chat interface with WebSocket handling logics - Add the chat interface on the UI - Add the framework of WebSocket handling logic in frontend and backend - To keep this PR manageable, the chat experience will be a simple echo of the user's message. No OpenAI dependency in this PR yet. - How to test: Verify if the frontend and backend can work together with simple echo'ed responses. #### PR3: Add LLM-agent-based chat experience - Add a simple single-agent based chat experience based on OpenAI Agents SDK - Add in-memory session management - Introduce dependencies on OpenAI and API keys, prompt loading, model configurations, etc. - How to test: Verify if inside Texera's frontend a user can chat with a OpenAI model with in-session conversation history. #### PR4: Add multi-agent system for Texera workflow generation - Enables the complete Texera Chat Assistant experience - Add a 3-agent orchestration system (Manager, Planner, Builder) - Add function calling capabilities - How to test: Verify if a workflow can be created by chatting with the assistant. #### PR5: Add deployment files - All previous PRs are to be tested locally. - This PR adds deployment-related files, including Dockerfile, K8s templates, etc. - How to test: Verify if the new microservice can be deployed using the added files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
