Zhuoxi2000 commented on issue #928: URL: https://github.com/apache/flink-agents/issues/928#issuecomment-5095796929
Thanks @piosystems — agreed on going with a dedicated connection. The shape I’m planning is: * **Java**: add `VLLMChatModelConnection` / `VLLMChatModelSetup` under the existing `integrations/chat-models/openai` module, similar to how Azure OpenAI is packaged today. This would reuse the OpenAI-compatible client, default `api_base_url` to `http://localhost:8000/v1`, and make `api_key` optional. * **Python**: add `flink_agents/integrations/chat_models/vllm/vllm_chat_model.py`, mirroring the existing `chat_models/azure` layout. * Add `ResourceName.ChatModel.VLLM_CONNECTION` / `VLLM_SETUP` and the Python variants. * Add a `vllm` alias in both `Aliases.java` and `aliases.py`. * Add docs and a YAML example, following #812. I’ll start with this direction. If a committer would prefer to scope it down to the alias-only approach, just let me know and I can adjust the alias would be included either way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
