PG1204 opened a new pull request, #8630:
URL: https://github.com/apache/texera/pull/8630
### What changes were proposed in this PR?
`zero-shot-image-classification` puts its labels in the payload as
`parameters.candidate_labels`, which only hf-inference understands. When the
operator falls back to a third-party chat provider, the request is rebuilt as
an image part plus a text part, and that text was just `prompt_value`, so the
provider received an image with no labels at all, and answered with a generic
caption instead of a classification, silently.
`_chat_content_for_task` now has a branch for the task that turns the labels
into an instruction ("Classify the image into exactly one of these labels: …"),
and the image chat branches use it for their text part instead of the bare
prompt. When no labels are configured it returns `prompt_value`, so nothing
else changes.
The branch is deliberately separate from the existing
`zero-shot-classification` one rather than merged into it: there `inputs` is
the text being classified, but here `inputs` is the base64 image, so reusing
that path would have inlined the whole image into the prompt. A test covers
that specifically.
Four call sites now route through the helper — the three chat branches in
`_call_provider` plus the model-author branch in `_post_with_fallback`. The
fourth is a no-op for its own tasks, but it makes the invariant uniform and
testable.
This is Part B of #7906 and completes it; Part A (#7920) fixed the response
side.
### Any related issues?
Closes #7906
### How was this PR tested?
Written test-first. The three new tests in `HuggingFaceCodegenBaseSpec`
failed against the current template, then passed once the helper branch and the
call sites were in place. The full Hugging Face suite is at 146 tests,
`PythonCodeRawInvalidTextSpec` py-compiles the generated Python for all 117
operators, and `scalafmtCheck` and `scalafix --check` are clean.
The emitted helper was also run directly, comparing this branch against
main: the labels now reach the prompt, the base64 image never appears in it,
empty or missing labels still fall back to the prompt value, and
`zero-shot-classification`, `question-answering`, `visual-question-answering`
and `text-generation` are unchanged.
### Was this PR authored or co-authored using generative AI tooling?
Yes, this PR was co-authored with Claude in compliance with ASF policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
