Davis-Zhang-Onehouse opened a new pull request, #20731: URL: https://github.com/apache/hudi/pull/20731
### Describe the issue this Pull Request addresses issue: #20730 Part of #20683, RFC-118 (#20684). Stacked on #20729: review only the last commit. Until the PRs below it merge, this PR also shows their commits. An embedding provider base (request rate limiting, rejection policy, availability checks and per-call statistics) and the OpenAI-compatible embedding client (OpenAI, Azure OpenAI v1 and Hugging Face TEI style endpoints) with its compute writer. ### Summary and Changelog - Provider base: request rate limiting, a rejection policy for provider errors, text-provider availability checks and per-call statistics. - `OpenAiCompatibleEmbeddingClient` for OpenAI, Azure OpenAI v1 and Hugging Face TEI style endpoints, with its compute writer and rate-limit handling. - 10 test classes (9 new). Tests use recorded response shapes under `unstructured/openai/`; no live endpoint is called. No code was copied from other projects. ### Impact The embedding stream can call OpenAI-compatible embedding endpoints. No change outside the new source. ### Risk Level low: new provider code of the new source. Verified on JDK 11 / Spark 3.5 / Scala 2.12 with 136 tests (none skipped), module compilation, checkstyle and RAT. ### Documentation Update New provider settings carry their config documentation; a website page follows once the chain lands. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
