aglinxinyuan opened a new issue, #6249:
URL: https://github.com/apache/texera/issues/6249
### Task Summary
Add a dedicated `QaRankingCodegenSpec.scala` that pins the payload/parse
Python-snippet contract of `QaRankingCodegen` — the Hugging Face codegen for
the question-answering / ranking task family. Mirror the existing
`TextGenCodegenSpec` (same package) as the template.
## Background
`QaRankingCodegen`
(`common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala`)
is an `object extends TaskCodegen`. It produces two Python source snippets —
`payloadPython(ctx)` and `parsePython(ctx)` — that are spliced into the
generated inference operator. It handles five HF pipeline tasks and, like every
codegen, must reference `self.*` attributes rather than inlining raw user
strings from the `CodegenContext`.
```scala
override val task: String = "question-answering"
override val tasks: Set[String] = Set(
"question-answering", "table-question-answering",
"zero-shot-classification", "sentence-similarity", "text-ranking"
)
```
## Behavior to pin
| Member | Contract |
| --- | --- |
| `task` | equals `"question-answering"` |
| `tasks` | equals exactly the 5-element set above |
| `payloadPython` | opens with `if task == "question-answering":`; has an
`elif` branch for table-QA, zero-shot, sentence-similarity and text-ranking;
ends with the `else: payload = {"inputs": prompt_value}` fallback |
| `parsePython` | the QA / table-QA branches read `body.get("answer", ...)`;
zero-shot / sentence-similarity / text-ranking return `json.dumps(body)` |
| no raw-value leakage | with sentinel `CodegenContext` string fields (e.g.
`contextColumn` / `candidateLabels` / `sentencesColumn` set to distinctive
`"MARKER_…"` values), neither snippet contains the sentinel — the code
references `self.CONTEXT_COLUMN` / `self.CANDIDATE_LABELS` /
`self.SENTENCES_COLUMN` instead |
Build the `CodegenContext` with a small `makeCtx` helper exactly like
`TextGenCodegenSpec` (set `task = "question-answering"` plus the QA-specific
columns). Assert on snippet structure / marker substrings, not on exact
whitespace.
## Scope
- New spec: `QaRankingCodegenSpec.scala` under
`common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/`.
- No production-code changes.
### Task Type
- [ ] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [x] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
