This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6972-bfe2b535c031cda0508bae6a9a6d2b7f62a886c6 in repository https://gitbox.apache.org/repos/asf/texera.git
commit 3f58aceacdca6a4657c38b64d692ad2e8e6f592d Author: Prateek Ganigi <[email protected]> AuthorDate: Tue Jul 28 14:19:11 2026 -0700 fix(workflow-operator): send provider-specific model id on HF text-gen chat fallback routes (#6972) ### What changes were proposed in this PR? Fixes two related bugs in the HuggingFace inference operator's provider-fallback loop (`_post_with_fallback` in the generated Python, emitted by `PythonCodegenBase.scala`) that break the `text-generation` / `image-text-to-text` path, which is the operator's default task. 1. **Wrong model id on provider-scoped chat routes.** The chat branch posted the same payload, carrying `"model": self.MODEL_ID` (the HF Hub ID), to every provider's route (`router.huggingface.co/{provider}/...`). Provider-scoped routes require the provider's *own* model name (`providerId`, e.g. the Hub's `Qwen/Qwen2.5-72B-Instruct` is a different string on Groq or Fireworks), which `_resolve_providers` already fetches and `_call_provider` already uses for the other task families. Providers whose internal name differs from the Hub ID rejected every request with 400/404, so fallback only succeeded when the names happened to match. The chat branch now posts a per-attempt copy with the model overridden: chat_payload = {**pipeline_payload, "model": provider_id} (a copy, not in-place mutation, pipeline_payload is reused for the next provider attempt). 2. **Malformed hf-inference chat URL.** The chat URL for hf-inference was built as `hf-inference/v1/chat/completions`, but hf-inference expects the model in the URL path: `hf-inference/models/{model-id}/v1/chat/completions` (the form this file already uses for pipeline tasks). Since hf-inference sorts first in `PROVIDER_COST_PRIORITY`, the cheapest provider failed on every row, wasting a doomed request (up to the 120 s timeout) per row before falling through. Net effect: text-gen rows now succeed on the cheapest live provider instead of failing with "All inference providers failed" or silently drifting to pricier providers. No behavior change for other task families - `_call_provider` already handled them correctly. ### Any related issues, documentation, discussions? Closes #6965 ### How was this PR tested? Added a generated-code test to HuggingFaceInferenceOpDescSpec ("send the provider-specific model id on provider-scoped chat routes") pinning both fixes: the corrected hf-inference chat URL, and the chat branch posting the per-provider payload copy (asserted as an anchored two-line block so pipeline routes that legitimately post pipeline_payload directly stay unaffected). Run: sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.huggingFace.*" Full HF package passes (123 tests, 12 suites), including PythonCodeRawInvalidTextSpec, which py_compiles the generated Python of all 117 Python operators, verifying the edited template still emits syntactically valid Python. scalafmt clean. ### Was this PR authored or co-authored using generative AI tooling? Co-authored with Claude Fable 5 in compliance with ASF. --- .../huggingFace/codegen/PythonCodegenBase.scala | 15 ++++++++++++--- .../huggingFace/HuggingFaceInferenceOpDescSpec.scala | 17 +++++++++++++++++ 2 files changed, 29 insertions(+), 3 deletions(-) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala index 7cd305bfb6..f2cf479afb 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala @@ -201,9 +201,18 @@ object PythonCodegenBase { | prov_task = prov.get("task", "") | try: | if self.TASK in ("text-generation", "image-text-to-text"): - | route = self.CHAT_ROUTES.get(provider_name, "v1/chat/completions") - | url = f"https://router.huggingface.co/{provider_name}/{route}"; - | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120) + | if provider_name == "hf-inference": + | # hf-inference expects the model in the URL path, like the + | # pipeline route below. + | url = f"https://router.huggingface.co/hf-inference/models/{self.MODEL_ID}/v1/chat/completions"; + | else: + | route = self.CHAT_ROUTES.get(provider_name, "v1/chat/completions") + | url = f"https://router.huggingface.co/{provider_name}/{route}"; + | # Provider-scoped routes need the provider's own model name + | # (providerId), not the HF Hub ID. Copy instead of mutating: + | # pipeline_payload is reused for the next provider attempt. + | chat_payload = {**pipeline_payload, "model": provider_id} + | resp = requests.post(url, headers=json_headers, json=chat_payload, timeout=120) | elif is_model_author and prov_task in ("image-to-text", "image-text-to-text") and provider_name not in ("zai-org",): | url = f"https://router.huggingface.co/{provider_name}/v1/chat/completions"; | img_b64 = "" diff --git a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala index 83f6239903..97bc9dc8ff 100644 --- a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala +++ b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala @@ -113,6 +113,23 @@ class HuggingFaceInferenceOpDescSpec extends AnyFlatSpec with Matchers { code should include("""body["choices"][0]["message"]["content"]""") } + it should "send the provider-specific model id on provider-scoped chat routes" in { + val code = makeDesc().generatePythonCode() + // hf-inference's chat-completions endpoint carries the model in the URL path. + code should include( + "https://router.huggingface.co/hf-inference/models/{self.MODEL_ID}/v1/chat/completions"; + ) + // The chat branch posts a per-provider copy of the payload with the + // provider's own model name (providerId) — the Hub ID is only valid on + // hf-inference itself. Matched as one block so the assertion is anchored + // to the chat branch: pipeline routes elsewhere legitimately post the + // shared pipeline_payload directly. + code should include( + "chat_payload = {**pipeline_payload, \"model\": provider_id}\n" + + " resp = requests.post(url, headers=json_headers, json=chat_payload, timeout=120)" + ) + } + it should "emit a runtime check that rejects malformed MODEL_ID values before any HF URL is built" in { val code = makeDesc().generatePythonCode()
