josix opened a new issue, #41336: URL: https://github.com/apache/airflow/issues/41336
### Description For generating a large number of responses from OpenAI's LLM, we can leverage the [Batch API](https://platform.openai.com/docs/guides/batch/overview) to send asynchronous groups of requests and wait for their completion when rapid response is not required. This approach can also help reduce costs in use cases such as prompt evaluations and applying prompts to each entity in large datasets. Supporting this functionality would be particularly useful for workflows in Airflow, which is commonly used for batch processing of different datasets. It would be great if the OpenAI provider could support the Batch API. ### Use case/motivation To support Batch API, the OpenAI Hook would need to include the following behaviors: 1. Support for OpenAI().files.create, which is already implemented. 1. Support for OpenAI().batches.create to submit a job for OpenAI to make predictions on the file. I think that would be nice if we have a new deferrable operator to submit the batch prediction and retrieve the results once finished, that would be helpful. ### Related issues _No response_ ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
