Croway opened a new pull request, #26601: URL: https://github.com/apache/camel/pull/26601
Adds the OpenAI Batch API to `camel-openai` ([CAMEL-24674](https://issues.apache.org/jira/browse/CAMEL-24674)). The Batch API runs requests offline at half the price of the synchronous API, against a separate rate limit, and returns the answers as a file within 24 hours. It fits work nobody waits for: overnight enrichment, embedding backfills, moderation backlogs and prompt regression runs. Everything between the upload and the result file is integration work, which is what this makes routable. ## Operations | Operation | Does | |---|---| | `batch` | Uploads the requests as a JSONL file and creates the batch | | `batch-retrieve` | Reports status and request counts, leaving the body untouched | | `batch-cancel` | Cancels a batch that is still running | | `batch-results` | Downloads the output file, or the error file with `batchResultsFile=error` | ## Building the input The body is either the JSONL itself (`File`, `Path`, `WrappedFile`, `InputStream`, `byte[]`, String), or a `Map` keyed by `custom_id`. For a map the component writes the envelope of every line (`method`, and `url` from `batchEndpoint`), and the value decides the request body: - a String is turned into a request built from the endpoint options (`/v1/chat/completions`, `/v1/responses`, `/v1/embeddings`, `/v1/moderations`), so a route holding prompts does not assemble API payloads itself - a `Map` or `JsonNode` is used as the request body as it is, which covers the other endpoints and per-line differences ```java from("direct:classify") .setBody(constant(Map.of("ticket-1", "I was charged twice", "ticket-2", "The app crashes"))) .to("openai:batch?batchEndpoint=/v1/chat/completions&model=gpt-4o-mini&systemMessage=Classify the ticket"); from("direct:collect") .noStreamCaching() .to("openai:batch-results") .split(body().tokenize("\n")).streaming() .setHeader("ticketId", jsonpath("$.custom_id")) .setBody(jsonpath("$.response.body.choices[0].message.content")) .to("sql:update ticket set category = :#${body} where id = :#${header.ticketId}") .end(); ``` ## Behaviour worth reviewing - The endpoint is validated against the eight endpoints the API supports **before** anything is uploaded, so a typo leaves no orphan file. - A create that fails deletes the file it just uploaded, attaching any delete failure as a suppressed exception. - Map bodies are written to a temporary file, so the SDK can retry the upload; a one-shot stream could not be resent. The file is deleted afterwards. - `batch-results` works for `completed`, `expired` and `cancelled`; it fails while the batch is still running, and reports the validation errors for a `failed` batch (also in `CamelOpenAIBatchErrors`). - A batch without failures has no error file, so the body is `null` and splits into nothing. - The results stream is closed when the exchange completes, and must be consumed inside the route. - `streaming`, `conversationMemory`, `mcpServer` and `tags` are rejected rather than silently ignored, since a batch line is a single request. - Every SDK call goes through `GenAiErrorSupport`. No GenAI span is recorded: creating a batch is not an inference call and `GenAiOperationName` has no value for it. - Result lines are model output, so the docs unmarshal them to a `Map` and never to a type the content chooses. ## Test infra `camel-test-infra-openai-mock` now serves the Files and Batch APIs: upload, retrieve and download of files, batch create/retrieve/cancel, a configurable status progression advancing one step per retrieve, and output/error files built from expectations matched by `custom_id`. `getBatchStore().getUploadedFile()` returns the JSONL a route produced. ## Verified - **On the mock:** 9 tests in `OpenAIBatchMockTest` — prompts uploaded as JSONL with the envelope and options applied, raw JSONL passed through unchanged, an unsupported endpoint rejected before upload, status and counts while polling, cancel, output streaming correlated by `custom_id`, the error file, results requested too early, and a missing batch id. The full `camel-openai` suite passes. - **Not verified:** nothing has run against the real Batch API or another implementation yet. An opt-in `OpenAIBatchExternalServiceIT` (Llama Stack in front of vLLM/Ollama, then OpenAI itself) is the next step, along with checking what partial results an expired or cancelled batch actually returns. Pre-existing and unrelated: `OpenAIEmbeddingsIT` fails locally because the Ollama model `granite-embedding:30m` is not pulled. ## Follow-up Generic Files API operations (`file-upload`, `file-content`, `file-delete`) are deliberately left out; they belong with Responses `input_file` and vector stores in their own ticket. Reacting to the `batch.completed` webhook instead of polling is CAMEL-24673. _Claude Code on behalf of Federico Mariani (Croway)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
