PG1204 commented on code in PR #8617:
URL: https://github.com/apache/texera/pull/8617#discussion_r4068220689
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/TextGenCodegenSpec.scala:
##########
@@ -74,6 +74,18 @@ class TextGenCodegenSpec extends AnyFlatSpec with Matchers {
out should include("content")
}
+ it should "degrade instead of raising when a chat response is malformed
(#8486)" in {
+ // This extraction was fully unguarded: a non-dict body, an empty "choices"
+ // list, or a choice missing "message"/"content" raised, and since parsing
+ // runs per row that aborted the whole run over one bad response. It now
+ // falls back to the raw JSON body, as the other codegens do.
+ val out = TextGenCodegen.parsePython(makeCtx())
+ out should include("""if isinstance(body, dict) and
body.get("choices"):""")
+ out should include("""body["choices"][0].get("message", {}).get("content",
json.dumps(body))""")
+ out should include("return json.dumps(body)")
+ out should not include ("""["message"]["content"]""")
Review Comment:
Same fix, HuggingFaceParseBehaviorSpec now executes the generated parser
rather than inspecting its text, covering text-generation and all five
QA/ranking tasks across valid, empty, missing, null and wrong-typed nested
shapes, and asserting malformed input returns the serialized body.
It follows the existing PythonCodeRawInvalidTextSpec pattern for locating an
interpreter, and cancels rather than fails where none is available.
ImageTaskCodegen has the same extractions from #7920 and will be routed through
the helper and added to this spec in a small follow-up PR, so that change
arrives with its own failing-first test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
