PG1204 commented on code in PR #8617:
URL: https://github.com/apache/texera/pull/8617#discussion_r4068219732
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegenSpec.scala:
##########
@@ -117,7 +117,19 @@ class QaRankingCodegenSpec extends AnyFlatSpec with
Matchers {
out should include("""body.get("answer"""")
// #7195: chat-completions responses (third-party providers) are read from
// choices[0].message.content, not the native {"answer": ...} shape.
- out should include("""body["choices"][0]["message"]["content"]""")
+ out should include("""body["choices"][0].get("message", {}).get("content",
json.dumps(body))""")
+ }
+
+ it should "degrade instead of raising when a chat response is malformed
(#8486)" in {
+ // parsePython runs per row, so indexing straight into
+ // choices[0]["message"]["content"] turned one malformed provider response
+ // into an aborted run: an empty "choices" list raises IndexError and a
+ // choice missing "message"/"content" raises KeyError. All three chat
+ // extractions now use a truthiness guard plus .get chaining, matching the
+ // native shapes beside them, which already degrade via json.dumps(body).
+ val out = QaRankingCodegen.parsePython(makeCtx())
+ out should not include ("""["message"]["content"]""")
+ out.split("""body\.get\("choices"\)""").length - 1 shouldBe 3
Review Comment:
Agreed that these assert that the source text changed, not that behavior
did. Added HuggingFaceParseBehaviorSpec, which renders the operator, lifts
_parse_response and its helpers out of the generated source, and runs them
under a real interpreter against 13 response shapes: valid, empty, missing,
null and wrong-typed, asserting the actual return value for every task in this
PR. The contract it pins: always returns a string, never raises, never None.
It has teeth: run against this PR's previous commit, 2 of its 4 tests fail
on exactly the AttributeError cases you identified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
