This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-11997-ffdffb98290f20f8a1b5725e72d3c31bb7a9c253 in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit fa66cce41587838581014dca039e8c249adeb219 Author: kuswardhanietidims-svg <[email protected]> AuthorDate: Sun Aug 30 04:22:32 2026 +0000 [Feature][CLI] Add OrcaRouter AI gateway LLM provider (#11997) Co-authored-by: kuswardhanietidims-svg <[email protected]> Co-authored-by: Claude <[email protected]> --- docs/en/ai-cli/overview.md | 2 +- docs/en/ai-cli/quickstart.md | 37 ++++ docs/zh/ai-cli/overview.md | 2 +- docs/zh/ai-cli/quickstart.md | 34 ++++ seatunnel-cli/README.md | 37 +++- seatunnel-cli/README.zh-CN.md | 32 +++- seatunnel-cli/env.example.sh | 11 ++ seatunnel-cli/seatunnel_cli/cli.py | 59 +++++- seatunnel-cli/seatunnel_cli/llm_provider.py | 96 ++++++++-- seatunnel-cli/setup.sh | 4 + seatunnel-cli/tests/test_cli_provider_routing.py | 13 +- .../tests/test_llm_provider_orcarouter.py | 200 +++++++++++++++++++++ 12 files changed, 498 insertions(+), 29 deletions(-) diff --git a/docs/en/ai-cli/overview.md b/docs/en/ai-cli/overview.md index bd505651be..6ebd1c1208 100644 --- a/docs/en/ai-cli/overview.md +++ b/docs/en/ai-cli/overview.md @@ -21,7 +21,7 @@ The AI CLI ships as the `seatunnel-cli` module inside the main SeaTunnel reposit ## Key Capabilities - **Natural language to config** — English or Chinese input, complete HOCON output -- **Multi-provider LLM** — AWS Bedrock (including OpenAI-family models via the `bedrock-mantle` endpoint), Anthropic API, OpenAI and compatible APIs +- **Multi-provider LLM** — AWS Bedrock (including OpenAI-family models via the `bedrock-mantle` endpoint), Anthropic API, OpenAI and compatible APIs, OrcaRouter AI gateway - **Multi-agent pipeline** — Planner → Config Generator → Validator → auto-fix, with up to 3 correction rounds - **Connector knowledge** — 150+ connectors with full option rules and value constraints, resolved from a live engine or bundled metadata - **Validation & repair** — local checks, engine `--check`/dry-run, and LLM-powered diagnosis-and-repair when `/check` or `/run` fails diff --git a/docs/en/ai-cli/quickstart.md b/docs/en/ai-cli/quickstart.md index a1ea10f88f..a513ec07b9 100644 --- a/docs/en/ai-cli/quickstart.md +++ b/docs/en/ai-cli/quickstart.md @@ -12,6 +12,7 @@ sidebar_position: 2 - **AWS Bedrock** — AWS credentials (profile, env vars, or IAM role) - **Anthropic API** — `ANTHROPIC_API_KEY` - **OpenAI API** (or compatible) — `OPENAI_API_KEY` + - **OrcaRouter** — `ORCAROUTER_API_KEY` - (Optional) a SeaTunnel installation for engine-level validation and job execution ## Install @@ -58,8 +59,44 @@ export ANTHROPIC_API_KEY=sk-ant-... export AI_PROVIDER=openai export OPENAI_API_KEY=sk-... # export OPENAI_BASE_URL=https://... # Azure OpenAI, DeepSeek, local vLLM, ... + +# Option D: OrcaRouter AI gateway +export AI_PROVIDER=orcarouter +export ORCAROUTER_API_KEY=orc_... +# Model IDs use a provider/model namespace (e.g. deepseek/deepseek-v4-pro); +# `orcarouter/auto` auto-grades and auto-routes each request. +# export ORCAROUTER_MODEL=orcarouter/auto +# export ORCAROUTER_SMALL_FAST_MODEL=orcarouter/auto +# export ORCAROUTER_ECHO_REASONING_CONTENT=true # optional: replay reasoning_content for reasoning models +``` + +### OrcaRouter AI gateway + +[OrcaRouter](https://www.orcarouter.ai) is an OpenAI-compatible AI gateway +that exposes many models — Claude, GPT, Gemini, DeepSeek, Qwen and more — +behind a single endpoint (`https://api.orcarouter.ai/v1`). Model IDs follow a +`provider/model` namespace, and the special `orcarouter/auto` model +automatically selects the best model per request. Configure it as a +first-class provider: + +```bash +# Requires the openai package (shares the ".[openai]" extra) +pip install -e ".[openai]" + +export AI_PROVIDER=orcarouter +export ORCAROUTER_API_KEY=orc_... +# export ORCAROUTER_MODEL=deepseek/deepseek-v4-pro # optional override +# export ORCAROUTER_SMALL_FAST_MODEL=orcarouter/auto # optional override +# export ORCAROUTER_ECHO_REASONING_CONTENT=true # optional: replay reasoning_content + +seatunnel "Sync MySQL users table to S3 Parquet" ``` +The provider speaks the OpenAI Chat Completions protocol, so it fully supports +the CLI's internal tool-calling loop (connector lookups during planning), +streaming output, multi-turn sessions, and reasoning-content replay for +compatible reasoning models. + ### bedrock-mantle: OpenAI-family models on Bedrock Some OpenAI models on Bedrock (e.g. `openai.gpt-5.6-terra`, `openai.gpt-5.6-sol`) diff --git a/docs/zh/ai-cli/overview.md b/docs/zh/ai-cli/overview.md index e53961264d..e09faf64b5 100644 --- a/docs/zh/ai-cli/overview.md +++ b/docs/zh/ai-cli/overview.md @@ -21,7 +21,7 @@ AI CLI 以 `seatunnel-cli` 模块的形式内置在 SeaTunnel 主仓库中，并 ## 核心能力 - **自然语言生成配置** —— 中英文输入，输出完整 HOCON 配置 -- **多 LLM 提供商** —— AWS Bedrock（含通过 `bedrock-mantle` 端点接入的 OpenAI 系模型）、Anthropic API、OpenAI 及兼容 API +- **多 LLM 提供商** —— AWS Bedrock（含通过 `bedrock-mantle` 端点接入的 OpenAI 系模型）、Anthropic API、OpenAI 及兼容 API、OrcaRouter AI 网关 - **多智能体流水线** —— Planner → 配置生成 → 校验 → 自动修复，最多 3 轮纠错 - **连接器知识库** —— 150+ 连接器的完整选项规则与取值约束，来自运行中引擎或内置元数据 - **校验与修复** —— 本地检查、引擎 `--check`/dry-run，`/check` 或 `/run` 失败时由 LLM 自动诊断修复 diff --git a/docs/zh/ai-cli/quickstart.md b/docs/zh/ai-cli/quickstart.md index d7b722e982..02806de4bc 100644 --- a/docs/zh/ai-cli/quickstart.md +++ b/docs/zh/ai-cli/quickstart.md @@ -12,6 +12,7 @@ sidebar_position: 2 - **AWS Bedrock** —— AWS 凭证（profile、环境变量或 IAM 角色） - **Anthropic API** —— `ANTHROPIC_API_KEY` - **OpenAI API**（或兼容 API）—— `OPENAI_API_KEY` + - **OrcaRouter** —— `ORCAROUTER_API_KEY` - （可选）SeaTunnel 安装目录，用于引擎级校验和作业执行 ## 安装 @@ -57,8 +58,41 @@ export ANTHROPIC_API_KEY=sk-ant-... export AI_PROVIDER=openai export OPENAI_API_KEY=sk-... # export OPENAI_BASE_URL=https://... # Azure OpenAI、DeepSeek、本地 vLLM 等 + +# 方式 D：OrcaRouter AI 网关 +export AI_PROVIDER=orcarouter +export ORCAROUTER_API_KEY=orc_... +# 模型 ID 使用 provider/model 命名空间（如 deepseek/deepseek-v4-pro）； +# `orcarouter/auto` 会自动评级并路由每个请求。 +# export ORCAROUTER_MODEL=orcarouter/auto +# export ORCAROUTER_SMALL_FAST_MODEL=orcarouter/auto +# export ORCAROUTER_ECHO_REASONING_CONTENT=true # 可选：保留并回传推理模型的 reasoning_content +``` + +### OrcaRouter AI 网关 + +[OrcaRouter](https://www.orcarouter.ai) 是一个 OpenAI 兼容的 AI 网关，在单个端点 +（`https://api.orcarouter.ai/v1`）之后暴露众多模型——Claude、GPT、Gemini、 +DeepSeek、Qwen 等。模型 ID 使用 `provider/model` 命名空间，特殊的 +`orcarouter/auto` 模型会自动为每个请求选择最佳模型。作为一等提供商配置： + +```bash +# 需要 openai 包（复用 ".[openai]" extra） +pip install -e ".[openai]" + +export AI_PROVIDER=orcarouter +export ORCAROUTER_API_KEY=orc_... +# export ORCAROUTER_MODEL=deepseek/deepseek-v4-pro # 可选覆盖 +# export ORCAROUTER_SMALL_FAST_MODEL=orcarouter/auto # 可选覆盖 +# export ORCAROUTER_ECHO_REASONING_CONTENT=true # 可选：回传 reasoning_content + +seatunnel "Sync MySQL users table to S3 Parquet" ``` +该提供商使用 OpenAI Chat Completions 协议，因此完全支持 CLI 内部的工具调用循环 +（规划期间的连接器查询）、流式输出、多轮会话，以及兼容推理模型的 +reasoning_content 回放。 + ### bedrock-mantle：Bedrock 上的 OpenAI 系模型 Bedrock 上的部分 OpenAI 模型（如 `openai.gpt-5.6-terra`、`openai.gpt-5.6-sol`） diff --git a/seatunnel-cli/README.md b/seatunnel-cli/README.md index de890b0964..97b8772665 100644 --- a/seatunnel-cli/README.md +++ b/seatunnel-cli/README.md @@ -7,7 +7,7 @@ Describe your data synchronization task in English or Chinese, and the CLI gener ## Features - **Natural Language to Config** -- Describe what you want in plain English or Chinese, get a valid SeaTunnel config -- **Multi-Provider LLM** -- AWS Bedrock, Anthropic API, OpenAI (and compatible APIs like Azure OpenAI) +- **Multi-Provider LLM** -- AWS Bedrock, Anthropic API, OpenAI (and compatible APIs like Azure OpenAI), OrcaRouter AI gateway - **Multi-Agent Pipeline** -- Planner -> Generator -> Validator -> Auto-fix, up to 3 correction rounds - **100+ Connectors** -- Full coverage of SeaTunnel's connector ecosystem with runtime metadata reflection - **Transform Metadata** -- Source, sink, and transform plugins use full option rules and value constraints during generation @@ -27,6 +27,7 @@ Describe your data synchronization task in English or Chinese, and the CLI gener - **AWS Bedrock** -- requires AWS credentials and `boto3` - **Anthropic API** -- requires `ANTHROPIC_API_KEY` and `anthropic` package - **OpenAI API** -- requires `OPENAI_API_KEY` and `openai` package + - **OrcaRouter** -- requires `ORCAROUTER_API_KEY` and `openai` package - (Optional) Running Apache SeaTunnel engine for live metadata and job execution > **Note:** When launched via `bin/seatunnel-ai.sh`, Python dependencies are > installed automatically on first run. No manual `pip install` needed. @@ -67,7 +68,7 @@ seatunnel --init ```bash pip install -e ".[bedrock]" # AWS Bedrock pip install -e ".[anthropic]" # Anthropic API -pip install -e ".[openai]" # OpenAI API +pip install -e ".[openai]" # OpenAI API / OrcaRouter pip install -e ".[all]" # All providers pip install -e ".[dev]" # Development (all providers + pytest, ruff) ``` @@ -165,6 +166,30 @@ export OPENAI_SMALL_FAST_MODEL=gpt-4o-mini # export OPENAI_ECHO_REASONING_CONTENT=true ``` +#### Option D: OrcaRouter AI Gateway + +```bash +export AI_PROVIDER=orcarouter +export ORCAROUTER_API_KEY=orc_... + +# Model overrides (optional) — OrcaRouter model IDs use a provider/model namespace, +# e.g. openai/gpt-5.5-pro, deepseek/deepseek-v4-pro, anthropic/claude-fable-5. +# The special model `orcarouter/auto` auto-grades and auto-routes each request. +# export ORCAROUTER_MODEL=orcarouter/auto +# export ORCAROUTER_SMALL_FAST_MODEL=orcarouter/auto +# export ORCAROUTER_ECHO_REASONING_CONTENT=true # optional: replay reasoning_content for reasoning models +``` + +Requires: `pip install -e ".[openai]"` (the `openai` package). + +[OrcaRouter](https://www.orcarouter.ai) is an OpenAI-compatible AI gateway +that exposes many models — including Claude, GPT, Gemini, DeepSeek and Qwen — +behind a single endpoint (`https://api.orcarouter.ai/v1`). Model IDs follow a +`provider/model` namespace, and the `orcarouter/auto` model automatically +selects the best model per request. The OrcaRouter provider speaks the OpenAI +Chat Completions protocol, so it inherits the same reasoning-content replay, +streaming and tool-calling support as the `openai` provider. + ### SEATUNNEL_HOME `SEATUNNEL_HOME` is the path to your Apache SeaTunnel engine installation. The CLI uses it to: @@ -207,16 +232,20 @@ When the engine is running, the CLI operates in **cluster mode** with live conne | Variable | Required | Default | Description | |----------|----------|---------|-------------| -| `AI_PROVIDER` | No | `bedrock` | LLM provider: `bedrock`, `bedrock-mantle`, `anthropic`, or `openai` | +| `AI_PROVIDER` | No | `bedrock` | LLM provider: `bedrock`, `bedrock-mantle`, `anthropic`, `openai`, or `orcarouter` | | `AWS_REGION` | Bedrock | `us-east-1` | AWS region for Bedrock | | `ANTHROPIC_API_KEY` | Anthropic | -- | Anthropic API key | | `OPENAI_API_KEY` | OpenAI | -- | OpenAI API key | | `OPENAI_BASE_URL` | No | -- | Custom endpoint for OpenAI-compatible APIs | | `OPENAI_ECHO_REASONING_CONTENT` | No | `true` | Preserve and replay `reasoning_content` for OpenAI-compatible reasoning models such as DeepSeek or GLM thinking mode | +| `ORCAROUTER_API_KEY` | OrcaRouter | -- | OrcaRouter API key | | `ANTHROPIC_MODEL` | No | Provider default | Override primary model ID | | `ANTHROPIC_SMALL_FAST_MODEL` | No | Provider default | Override fast model ID | | `OPENAI_MODEL` | No | `gpt-4o` | Primary model for OpenAI provider | | `OPENAI_SMALL_FAST_MODEL` | No | `gpt-4o-mini` | Fast model for OpenAI provider | +| `ORCAROUTER_MODEL` | No | `orcarouter/auto` | Primary model for OrcaRouter provider (provider/model namespace) | +| `ORCAROUTER_SMALL_FAST_MODEL` | No | `orcarouter/auto` | Fast model for OrcaRouter provider | +| `ORCAROUTER_ECHO_REASONING_CONTENT` | No | `true` | Preserve and replay `reasoning_content` for OpenAI-compatible reasoning models (parity with `OPENAI_ECHO_REASONING_CONTENT`) | | `SEATUNNEL_HOME` | No | Auto-detect | SeaTunnel installation directory. Auto-detected in distribution tarball; set manually for source install | | `SEATUNNEL_API_BASE` | No | `http://localhost:5801` | SeaTunnel REST API endpoint | | `SEATUNNEL_CLI_DATA` | No | `<cli-package>/.data/` | Override CLI data directory (sessions, memory, config) | @@ -256,7 +285,7 @@ Positional: Options: -o, --output PATH Save generated config to file - --provider PROVIDER LLM provider: bedrock | anthropic | openai + --provider PROVIDER LLM provider: bedrock | bedrock-mantle | anthropic | openai | orcarouter --model MODEL Override primary model ID --fast-model MODEL Override fast model ID --sync-catalog PATH Regenerate connector catalog from SeaTunnel source diff --git a/seatunnel-cli/README.zh-CN.md b/seatunnel-cli/README.zh-CN.md index 0b0f97f57b..54d68339e1 100644 --- a/seatunnel-cli/README.zh-CN.md +++ b/seatunnel-cli/README.zh-CN.md @@ -7,7 +7,7 @@ ## 功能特性 - **自然语言转配置** -- 用中文或英文描述需求，即可获得有效的 SeaTunnel 配置 -- **多 LLM 提供商** -- 支持 AWS Bedrock、Anthropic API、OpenAI（及兼容 API，如 Azure OpenAI） +- **多 LLM 提供商** -- 支持 AWS Bedrock、Anthropic API、OpenAI（及兼容 API，如 Azure OpenAI）、OrcaRouter AI 网关 - **多智能体流水线** -- 规划器 -> 生成器 -> 校验器 -> 自动修复，最多 3 轮纠错 - **100+ 连接器** -- 全面覆盖 SeaTunnel 连接器生态，支持运行时元数据反射 - **Transform 元数据** -- Source、Sink 和 Transform 插件在生成配置时都支持完整选项规则和值约束 @@ -136,6 +136,28 @@ export OPENAI_SMALL_FAST_MODEL=gpt-4o-mini # export OPENAI_ECHO_REASONING_CONTENT=true ``` +#### 方案 D：OrcaRouter AI 网关 + +```bash +export AI_PROVIDER=orcarouter +export ORCAROUTER_API_KEY=orc_... + +# 模型覆盖（可选）——OrcaRouter 模型 ID 使用 provider/model 命名空间， +# 例如 openai/gpt-5.5-pro、deepseek/deepseek-v4-pro、anthropic/claude-fable-5。 +# 特殊模型 `orcarouter/auto` 会自动评级并路由每个请求。 +# export ORCAROUTER_MODEL=orcarouter/auto +# export ORCAROUTER_SMALL_FAST_MODEL=orcarouter/auto +# export ORCAROUTER_ECHO_REASONING_CONTENT=true # 可选：保留并回传推理模型的 reasoning_content +``` + +需要：`pip install -e ".[openai]"`（`openai` 包）。 + +[OrcaRouter](https://www.orcarouter.ai) 是一个 OpenAI 兼容的 AI 网关，在单个端点 +（`https://api.orcarouter.ai/v1`）之后暴露众多模型——Claude、GPT、Gemini、 +DeepSeek、Qwen 等。模型 ID 使用 `provider/model` 命名空间，`orcarouter/auto` +模型会自动为每个请求选择最佳模型。OrcaRouter 提供商使用 OpenAI Chat Completions +协议，因此与 `openai` 提供商一样支持 reasoning_content 回放、流式输出和工具调用。 + ### SEATUNNEL_HOME `SEATUNNEL_HOME` 是 Apache SeaTunnel 引擎的安装路径。CLI 使用它来： @@ -178,16 +200,20 @@ export SEATUNNEL_API_BASE=http://localhost:5801 # 默认值 | 变量 | 是否必需 | 默认值 | 说明 | |------|---------|--------|------| -| `AI_PROVIDER` | 否 | `bedrock` | LLM 提供商：`bedrock`、`anthropic` 或 `openai` | +| `AI_PROVIDER` | 否 | `bedrock` | LLM 提供商：`bedrock`、`bedrock-mantle`、`anthropic`、`openai` 或 `orcarouter` | | `AWS_REGION` | Bedrock 必需 | `us-east-1` | Bedrock 使用的 AWS 区域 | | `ANTHROPIC_API_KEY` | Anthropic 必需 | -- | Anthropic API 密钥 | | `OPENAI_API_KEY` | OpenAI 必需 | -- | OpenAI API 密钥 | | `OPENAI_BASE_URL` | 否 | -- | OpenAI 兼容 API 的自定义端点 | | `OPENAI_ECHO_REASONING_CONTENT` | 否 | `true` | 为 DeepSeek、GLM 思考模式等 OpenAI 兼容推理模型保留并回传 `reasoning_content` | +| `ORCAROUTER_API_KEY` | OrcaRouter 必需 | -- | OrcaRouter API 密钥 | | `ANTHROPIC_MODEL` | 否 | 提供商默认值 | 覆盖主模型 ID | | `ANTHROPIC_SMALL_FAST_MODEL` | 否 | 提供商默认值 | 覆盖快速模型 ID | | `OPENAI_MODEL` | 否 | `gpt-4o` | OpenAI 提供商的主模型 | | `OPENAI_SMALL_FAST_MODEL` | 否 | `gpt-4o-mini` | OpenAI 提供商的快速模型 | +| `ORCAROUTER_MODEL` | 否 | `orcarouter/auto` | OrcaRouter 提供商的主模型（provider/model 命名空间） | +| `ORCAROUTER_SMALL_FAST_MODEL` | 否 | `orcarouter/auto` | OrcaRouter 提供商的快速模型 | +| `ORCAROUTER_ECHO_REASONING_CONTENT` | 否 | `true` | 保留并回传 `reasoning_content`（与 `OPENAI_ECHO_REASONING_CONTENT` 对齐） | | `SEATUNNEL_HOME` | 否 | 自动检测 | SeaTunnel 安装目录。发行版压缩包中自动检测；源码安装需手动设置 | | `SEATUNNEL_API_BASE` | 否 | `http://localhost:5801` | SeaTunnel REST API 端点 | | `SEATUNNEL_CLI_DATA` | 否 | `<cli-package>/.data/` | 覆盖 CLI 数据目录（会话、记忆、配置） | @@ -227,7 +253,7 @@ seatunnel [request] [options] 选项： -o, --output PATH 将生成的配置保存到文件 - --provider PROVIDER LLM 提供商：bedrock | anthropic | openai + --provider PROVIDER LLM 提供商：bedrock | bedrock-mantle | anthropic | openai | orcarouter --model MODEL 覆盖主模型 ID --fast-model MODEL 覆盖快速模型 ID --sync-catalog PATH 从 SeaTunnel 源码重新生成连接器目录 diff --git a/seatunnel-cli/env.example.sh b/seatunnel-cli/env.example.sh index c684257df4..c53c4e5f57 100755 --- a/seatunnel-cli/env.example.sh +++ b/seatunnel-cli/env.example.sh @@ -29,6 +29,7 @@ # export AI_PROVIDER=bedrock-mantle # Option C2: OpenAI-family models on Bedrock # # (GPT-5.6 Terra/Sol; needs ".[bedrock-mantle]" extra; # # model via OPENAI_MODEL, e.g. openai.gpt-5.6-terra) +# export AI_PROVIDER=orcarouter # Option D: OrcaRouter AI gateway (needs ".[openai]" extra) # ─── Option A: Anthropic API (AI_PROVIDER=anthropic) ─── # export ANTHROPIC_API_KEY=sk-ant-... @@ -49,6 +50,16 @@ # export AWS_ACCESS_KEY_ID=... # export AWS_SECRET_ACCESS_KEY=... +# ─── Option D: OrcaRouter AI gateway (AI_PROVIDER=orcarouter) ─── +# OpenAI-compatible gateway: many models behind one endpoint, model IDs use a +# provider/model namespace (e.g. deepseek/deepseek-v4-pro, openai/gpt-5.5-pro). +# The special model `orcarouter/auto` auto-grades and auto-routes each request. +# Requires: pip install -e ".[openai]" +# export ORCAROUTER_API_KEY=orc_... +# export ORCAROUTER_MODEL=orcarouter/auto # optional override +# export ORCAROUTER_SMALL_FAST_MODEL=orcarouter/auto # optional override +# export ORCAROUTER_ECHO_REASONING_CONTENT=true # optional: keep true to replay reasoning_content for reasoning models + # ─── SeaTunnel Engine (optional) ─── # export SEATUNNEL_HOME=/path/to/seatunnel # export SEATUNNEL_API_BASE=http://localhost:5801 diff --git a/seatunnel-cli/seatunnel_cli/cli.py b/seatunnel-cli/seatunnel_cli/cli.py index 61988744af..89f8d7c028 100644 --- a/seatunnel-cli/seatunnel_cli/cli.py +++ b/seatunnel-cli/seatunnel_cli/cli.py @@ -514,20 +514,23 @@ class SeaTunnelCLI: console.print(" [bold]4[/bold]. bedrock-mantle — OpenAI-family models on Bedrock (GPT-5.6 Terra/Sol)") console.print(" Requires: AWS credentials + pip install \".[bedrock-mantle]\"") console.print(" Note: Responses-API-only models on the bedrock-mantle endpoint\n") + console.print(" [bold]5[/bold]. orcarouter — OrcaRouter AI gateway (OpenAI-compatible)") + console.print(" Requires: ORCAROUTER_API_KEY + pip install \".[openai]\"") + console.print(" Note: many models behind one endpoint, e.g. orcarouter/auto\n") try: - choice = pt_prompt(" Enter your choice (1/2/3/4): ").strip().lower() + choice = pt_prompt(" Enter your choice (1/2/3/4/5): ").strip().lower() except (EOFError, KeyboardInterrupt): console.print("\n Setup cancelled.", style="warning") return choice_map = {"1": "anthropic", "2": "openai", "3": "bedrock", - "4": "bedrock-mantle"} + "4": "bedrock-mantle", "5": "orcarouter"} choice = choice_map.get(choice, choice) if not choice or choice not in _PROVIDERS: console.print( - f" [error]Invalid choice: '{choice}'. Please enter 1, 2, 3, or 4.[/error]" + f" [error]Invalid choice: '{choice}'. Please enter 1, 2, 3, 4, or 5.[/error]" ) return @@ -603,6 +606,30 @@ class SeaTunnelCLI: config.setdefault("settings", {})["openai_base_url"] = base_url console.print(f" Base URL set: [bold]{base_url}[/bold]") + elif choice == "orcarouter": + existing = os.environ.get("ORCAROUTER_API_KEY") + if existing: + masked = existing[:7] + "..." + existing[-4:] if len(existing) > 15 else "***" + console.print(f" ORCAROUTER_API_KEY: [bold green]detected[/bold green] ({masked})") + else: + console.print(" ORCAROUTER_API_KEY not found in environment.\n") + console.print( + " [dim]Persistent (recommended): add to ~/.zshrc or ~/.bashrc:[/dim]\n" + " export ORCAROUTER_API_KEY=orc_...\n" + " [dim]Get a key: https://www.orcarouter.ai[/dim]\n";, + ) + try: + key_input = pt_prompt( + " Enter API key for this session (or Enter to skip): ", + ).strip() + except (EOFError, KeyboardInterrupt): + key_input = "" + if key_input: + os.environ["ORCAROUTER_API_KEY"] = key_input + console.print( + " [success]Key set for this session (NOT saved to disk).[/success]" + ) + elif choice in ("bedrock", "bedrock-mantle"): console.print(" AWS Bedrock requires AWS credentials.\n") if choice == "bedrock-mantle": @@ -665,6 +692,11 @@ class SeaTunnelCLI: default_fast = "gpt-4o-mini" model_env = "OPENAI_MODEL" fast_env = "OPENAI_SMALL_FAST_MODEL" + elif choice == "orcarouter": + default_model = "orcarouter/auto" + default_fast = "orcarouter/auto" + model_env = "ORCAROUTER_MODEL" + fast_env = "ORCAROUTER_SMALL_FAST_MODEL" elif choice == "bedrock-mantle": default_model = "openai.gpt-5.6-terra" default_fast = "openai.gpt-5.6-terra" @@ -881,6 +913,15 @@ class SeaTunnelCLI: base_url = os.environ.get("OPENAI_BASE_URL") if base_url: self.console.print(f" Base URL: [bold]{base_url}[/bold]", style="info") + elif provider_name == "orcarouter": + if os.environ.get("ORCAROUTER_API_KEY"): + self.console.print(" API key: [bold green]configured[/bold green]", style="info") + else: + creds_ok = False + self.console.print("[error]ORCAROUTER_API_KEY not set.[/error]") + self.console.print( + " Base URL: [bold]https://api.orcarouter.ai/v1[/bold]";, style="info" + ) self.console.print(f" Model: [bold]{provider.model_id}[/bold]", style="info") self.console.print(f" Fast model: [bold]{provider.fast_model_id}[/bold]", style="info") @@ -1527,7 +1568,7 @@ def main(): ) parser.add_argument( "--provider", - choices=["bedrock", "bedrock-mantle", "anthropic", "openai"], + choices=["bedrock", "bedrock-mantle", "anthropic", "openai", "orcarouter"], help="LLM provider (overrides AI_PROVIDER env var and config.json)", ) parser.add_argument( @@ -1579,17 +1620,21 @@ def main(): if args.provider: os.environ["AI_PROVIDER"] = args.provider # Providers speaking the OpenAI protocol read OPENAI_MODEL*; - # bedrock/anthropic read ANTHROPIC_MODEL*. + # bedrock/anthropic read ANTHROPIC_MODEL*; orcarouter has its own. _OPENAI_FAMILY = ("openai", "bedrock-mantle") if args.model: provider = os.environ.get("AI_PROVIDER", "").lower() - if provider in _OPENAI_FAMILY: + if provider == "orcarouter": + os.environ["ORCAROUTER_MODEL"] = args.model + elif provider in _OPENAI_FAMILY: os.environ["OPENAI_MODEL"] = args.model else: os.environ["ANTHROPIC_MODEL"] = args.model if args.fast_model: provider = os.environ.get("AI_PROVIDER", "").lower() - if provider in _OPENAI_FAMILY: + if provider == "orcarouter": + os.environ["ORCAROUTER_SMALL_FAST_MODEL"] = args.fast_model + elif provider in _OPENAI_FAMILY: os.environ["OPENAI_SMALL_FAST_MODEL"] = args.fast_model else: os.environ["ANTHROPIC_SMALL_FAST_MODEL"] = args.fast_model diff --git a/seatunnel-cli/seatunnel_cli/llm_provider.py b/seatunnel-cli/seatunnel_cli/llm_provider.py index 29a5c13384..a15fb43dfe 100644 --- a/seatunnel-cli/seatunnel_cli/llm_provider.py +++ b/seatunnel-cli/seatunnel_cli/llm_provider.py @@ -21,6 +21,7 @@ Supports multiple backends while presenting a unified interface: - bedrock : AWS Bedrock Converse API (Claude models) - anthropic: Anthropic Messages API (direct) - openai : OpenAI Chat Completions API + - orcarouter: OrcaRouter AI gateway (OpenAI Chat Completions API) All providers normalize their responses to a common internal format so that the agent layer (agents.py) needs no provider-specific code. @@ -1250,6 +1251,66 @@ class BedrockMantleProvider(LLMProvider): } +# ─── OrcaRouter Provider ─── + +class OrcaRouterProvider(OpenAIProvider): + """OrcaRouter AI gateway provider (OpenAI Chat Completions API). + + OrcaRouter is an OpenAI-compatible gateway that exposes many models (and + provider/model routing namespaces) behind one endpoint. Models are + addressed as ``provider/model``, e.g. ``orcarouter/auto`` routes and + grades automatically. Defaults to the ``orcarouter/auto`` model, which + automatically selects the best model for each request. + + Since OrcaRouter speaks the OpenAI Chat Completions protocol, this + provider mirrors :class:`OpenAIProvider` and only customizes the base + URL, the API key environment variable, and the default model. + """ + + #: OrcaRouter's OpenAI-compatible base URL. + DEFAULT_BASE_URL = "https://api.orcarouter.ai/v1"; + #: Default model: auto-routes to the best model for the request. + DEFAULT_MODEL = "orcarouter/auto" + + def __init__(self): + try: + import openai + except ImportError: + raise ImportError( + "openai package required for AI_PROVIDER=orcarouter. " + "Install it: pip install openai" + ) + + api_key = os.environ.get("ORCAROUTER_API_KEY") + if not api_key: + raise ValueError( + "ORCAROUTER_API_KEY environment variable is required for " + "AI_PROVIDER=orcarouter" + ) + + self._model_id = os.environ.get( + "ORCAROUTER_MODEL", + os.environ.get("OPENAI_MODEL", self.DEFAULT_MODEL)) + self._fast_model_id = os.environ.get( + "ORCAROUTER_SMALL_FAST_MODEL", + os.environ.get("OPENAI_SMALL_FAST_MODEL", self._model_id)) + self._client = openai.OpenAI(api_key=api_key, base_url=self.DEFAULT_BASE_URL) + self._echo_reasoning_content = _env_bool( + "ORCAROUTER_ECHO_REASONING_CONTENT", True) + + @property + def provider_name(self) -> str: + return "orcarouter" + + @property + def model_id(self) -> str: + return self._model_id + + @property + def fast_model_id(self) -> str: + return self._fast_model_id + + # ─── Config file ─── @@ -1286,11 +1347,15 @@ def _auto_detect_provider() -> str | None: if os.environ.get("ANTHROPIC_API_KEY"): return "anthropic" - # 2. OpenAI API key + # 2. OrcaRouter gateway key + if os.environ.get("ORCAROUTER_API_KEY"): + return "orcarouter" + + # 3. OpenAI API key if os.environ.get("OPENAI_API_KEY"): return "openai" - # 3. AWS credentials (for Bedrock) + # 4. AWS credentials (for Bedrock) if os.environ.get("AWS_ACCESS_KEY_ID") or os.environ.get("AWS_PROFILE"): return "bedrock" try: @@ -1312,6 +1377,7 @@ _PROVIDERS = { "bedrock-mantle": BedrockMantleProvider, "anthropic": AnthropicProvider, "openai": OpenAIProvider, + "orcarouter": OrcaRouterProvider, } @@ -1354,16 +1420,22 @@ def create_provider(provider: str | None = None) -> LLMProvider: # Apply model overrides from config file if name and "models" in config: model_config = config["models"].get(name, {}) - if model_config.get("model") and not os.environ.get("ANTHROPIC_MODEL") and not os.environ.get("OPENAI_MODEL"): - if name in ("openai", "bedrock-mantle"): - os.environ.setdefault("OPENAI_MODEL", model_config["model"]) - else: - os.environ.setdefault("ANTHROPIC_MODEL", model_config["model"]) - if model_config.get("fast_model") and not os.environ.get("ANTHROPIC_SMALL_FAST_MODEL") and not os.environ.get("OPENAI_SMALL_FAST_MODEL"): - if name in ("openai", "bedrock-mantle"): - os.environ.setdefault("OPENAI_SMALL_FAST_MODEL", model_config["fast_model"]) - else: - os.environ.setdefault("ANTHROPIC_SMALL_FAST_MODEL", model_config["fast_model"]) + if model_config.get("model"): + if name == "orcarouter": + os.environ.setdefault("ORCAROUTER_MODEL", model_config["model"]) + elif not os.environ.get("ANTHROPIC_MODEL") and not os.environ.get("OPENAI_MODEL"): + if name in ("openai", "bedrock-mantle"): + os.environ.setdefault("OPENAI_MODEL", model_config["model"]) + else: + os.environ.setdefault("ANTHROPIC_MODEL", model_config["model"]) + if model_config.get("fast_model"): + if name == "orcarouter": + os.environ.setdefault("ORCAROUTER_SMALL_FAST_MODEL", model_config["fast_model"]) + elif not os.environ.get("ANTHROPIC_SMALL_FAST_MODEL") and not os.environ.get("OPENAI_SMALL_FAST_MODEL"): + if name in ("openai", "bedrock-mantle"): + os.environ.setdefault("OPENAI_SMALL_FAST_MODEL", model_config["fast_model"]) + else: + os.environ.setdefault("ANTHROPIC_SMALL_FAST_MODEL", model_config["fast_model"]) # 4. Auto-detect if not name: diff --git a/seatunnel-cli/setup.sh b/seatunnel-cli/setup.sh index 833d045d18..07703bdde6 100644 --- a/seatunnel-cli/setup.sh +++ b/seatunnel-cli/setup.sh @@ -73,6 +73,10 @@ echo " # AWS Bedrock" echo " export AI_PROVIDER=bedrock" echo " export AWS_REGION=us-east-1" echo "" +echo " # OrcaRouter AI gateway (OpenAI-compatible)" +echo " export AI_PROVIDER=orcarouter" +echo " export ORCAROUTER_API_KEY=orc_..." +echo "" echo " 3. Run the CLI:" echo "" echo " seatunnel # Interactive mode" diff --git a/seatunnel-cli/tests/test_cli_provider_routing.py b/seatunnel-cli/tests/test_cli_provider_routing.py index 39531fdbbc..1b459c9606 100644 --- a/seatunnel-cli/tests/test_cli_provider_routing.py +++ b/seatunnel-cli/tests/test_cli_provider_routing.py @@ -40,6 +40,7 @@ def _run_main_until_provider(argv): captured["AI_PROVIDER"] = os.environ.get("AI_PROVIDER") captured["OPENAI_MODEL"] = os.environ.get("OPENAI_MODEL") captured["ANTHROPIC_MODEL"] = os.environ.get("ANTHROPIC_MODEL") + captured["ORCAROUTER_MODEL"] = os.environ.get("ORCAROUTER_MODEL") raise _Stop() with mock.patch.object(sys, "argv", ["seatunnel"] + argv), \ @@ -53,7 +54,8 @@ def _run_main_until_provider(argv): def _clean_env(): saved = {k: os.environ.pop(k, None) for k in ("AI_PROVIDER", "OPENAI_MODEL", "ANTHROPIC_MODEL", - "OPENAI_SMALL_FAST_MODEL", "ANTHROPIC_SMALL_FAST_MODEL")} + "OPENAI_SMALL_FAST_MODEL", "ANTHROPIC_SMALL_FAST_MODEL", + "ORCAROUTER_MODEL", "ORCAROUTER_SMALL_FAST_MODEL")} yield for k, v in saved.items(): if v is None: @@ -77,6 +79,15 @@ def test_bedrock_still_routes_anthropic_model(): assert captured["OPENAI_MODEL"] is None +def test_orcarouter_accepted_by_argparse_and_routes_own_model(): + captured = _run_main_until_provider( + ["--provider", "orcarouter", "--model", "deepseek/deepseek-v4-pro", "hi"]) + assert captured["AI_PROVIDER"] == "orcarouter" + assert captured["ORCAROUTER_MODEL"] == "deepseek/deepseek-v4-pro" + assert captured["OPENAI_MODEL"] is None + assert captured["ANTHROPIC_MODEL"] is None + + def test_unknown_provider_rejected(): from seatunnel_cli import cli with mock.patch.object(sys, "argv", diff --git a/seatunnel-cli/tests/test_llm_provider_orcarouter.py b/seatunnel-cli/tests/test_llm_provider_orcarouter.py new file mode 100644 index 0000000000..50af2280d2 --- /dev/null +++ b/seatunnel-cli/tests/test_llm_provider_orcarouter.py @@ -0,0 +1,200 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import unittest +from types import SimpleNamespace +from unittest import mock + +from seatunnel_cli.llm_provider import ( + LLMProvider, + OrcaRouterProvider, +) + + +class _FakeOrcaRouterCompletions: + def __init__(self, stream): + self.stream = stream + self.kwargs = None + + def create(self, **kwargs): + self.kwargs = kwargs + return self.stream + + +class _FakeOrcaRouterClient: + def __init__(self, stream): + self.completions = _FakeOrcaRouterCompletions(stream) + self.chat = SimpleNamespace(completions=self.completions) + + +def _chunk(delta, finish_reason=None): + return SimpleNamespace( + choices=[ + SimpleNamespace( + delta=delta, + finish_reason=finish_reason, + ) + ] + ) + + +class OrcaRouterProviderTest(unittest.TestCase): + def test_default_model_and_base_url(self): + provider = OrcaRouterProvider.__new__(OrcaRouterProvider) + self.assertEqual(provider.provider_name, "orcarouter") + self.assertEqual(OrcaRouterProvider.DEFAULT_BASE_URL, "https://api.orcarouter.ai/v1";) + self.assertEqual(OrcaRouterProvider.DEFAULT_MODEL, "orcarouter/auto") + + def test_init_requires_api_key(self): + with mock.patch.dict("os.environ", {}, clear=True): + from seatunnel_cli import llm_provider + with self.assertRaises(ValueError): + llm_provider.OrcaRouterProvider() + + def test_echo_reasoning_content_env_override(self): + with mock.patch.dict( + "os.environ", + {"ORCAROUTER_API_KEY": "orc_test", "ORCAROUTER_ECHO_REASONING_CONTENT": "false"}, + clear=False, + ): + from seatunnel_cli import llm_provider + provider = llm_provider.OrcaRouterProvider() + self.assertFalse(provider._echo_reasoning_content) + + with mock.patch.dict( + "os.environ", + {"ORCAROUTER_API_KEY": "orc_test", "ORCAROUTER_ECHO_REASONING_CONTENT": "true"}, + clear=False, + ): + from seatunnel_cli import llm_provider + provider = llm_provider.OrcaRouterProvider() + self.assertTrue(provider._echo_reasoning_content) + + # Defaults to True when unset (parity with OPENAI_ECHO_REASONING_CONTENT). + with mock.patch.dict("os.environ", {"ORCAROUTER_API_KEY": "orc_test"}, clear=False): + from seatunnel_cli import llm_provider + provider = llm_provider.OrcaRouterProvider() + self.assertTrue(provider._echo_reasoning_content) + + def test_stream_uses_openai_protocol_and_collects_text(self): + provider = OrcaRouterProvider.__new__(OrcaRouterProvider) + provider._model_id = "deepseek/deepseek-v4-pro" + provider._client = _FakeOrcaRouterClient( + [ + _chunk(SimpleNamespace(content="Sync ", tool_calls=None)), + _chunk(SimpleNamespace(content="users to S3", tool_calls=None)), + _chunk( + SimpleNamespace(content=None, tool_calls=None), + finish_reason="stop", + ), + ] + ) + + events = list( + provider.chat_stream( + messages=[ + { + "role": "user", + "content": [{"text": "sync mysql to s3"}], + } + ] + ) + ) + response = LLMProvider.collect_stream(events) + + self.assertEqual( + response["output"]["message"]["content"], + [{"text": "Sync users to S3"}], + ) + # The default model is used when no override is passed. + self.assertEqual( + provider._client.completions.kwargs["model"], "deepseek/deepseek-v4-pro" + ) + + def test_chat_round_trip(self): + provider = OrcaRouterProvider.__new__(OrcaRouterProvider) + provider._model_id = "orcarouter/auto" + response = SimpleNamespace( + choices=[ + SimpleNamespace( + message=SimpleNamespace( + content="PLAN: use Jdbc", + tool_calls=None, + ), + finish_reason="stop", + ) + ] + ) + provider._client = _FakeOrcaRouterClient(response) + + result = provider.chat( + messages=[ + { + "role": "user", + "content": [{"text": "sync oracle to iceberg"}], + } + ] + ) + + self.assertEqual( + result["output"]["message"]["content"], + [{"text": "PLAN: use Jdbc"}], + ) + self.assertEqual(result["stopReason"], "end_turn") + + def test_config_model_override_is_independent(self): + """Config-file model overrides apply even when unrelated model env + vars (ANTHROPIC_MODEL/OPENAI_MODEL) are set elsewhere.""" + import os + import tempfile + + from seatunnel_cli import llm_provider + + tmp = tempfile.TemporaryDirectory() + original = llm_provider.get_config_path + + def fake_get_config_path(): + return os.path.join(tmp.name, "config.json") + + llm_provider.get_config_path = fake_get_config_path + try: + with mock.patch.dict( + "os.environ", + {"ORCAROUTER_API_KEY": "orc_test", "ANTHROPIC_MODEL": "claude-other"}, + clear=False, + ): + llm_provider.save_config( + { + "provider": "orcarouter", + "models": { + "orcarouter": { + "model": "deepseek/deepseek-v4-pro", + "fast_model": "qwen/qwen3.8-flash", + } + }, + } + ) + provider = llm_provider.create_provider() + self.assertEqual(provider.model_id, "deepseek/deepseek-v4-pro") + self.assertEqual(provider.fast_model_id, "qwen/qwen3.8-flash") + finally: + llm_provider.get_config_path = original + tmp.cleanup() + + +if __name__ == "__main__": + unittest.main()
