The GitHub Actions job "Required Checks" on texera.git/main has failed. Run started by GitHub user github-merge-queue[bot] (triggered by github-merge-queue[bot]).
Head commit for run: 5b704c39d1535dd39283f5a3ef020791a567d1a0 / Eugene Gu <[email protected]> test(workflow-compiler): Extend WorkflowCompilerSpec to cover the Python code-generation error path (#7649) ### What changes were proposed in this PR? `WorkflowCompiler` scans every Python-based physical operator's generated code for the `#EXCEPTION DURING CODE GENERATION:` marker that `PythonOperatorDescriptor` embeds when an operator's `generatePythonCode` throws. On a hit it either appends a `RuntimeException("Operator is not configured properly: ...")` to the caller's error list (the editing-time, lenient path) or throws it immediately when no error list was given (the pre-execution, strict path). `WorkflowCompilerSpec`'s 15 tests never exercised either branch; the only existing test of the marker is the producer-side `PythonOperatorDescriptorSpec`, which asserts the marker is written but not that the compiler reacts to it. This adds 7 tests to `WorkflowCompilerSpec` (15 → 22). Test-only: no production file is touched, and none of the 15 existing tests is modified — the diff removes exactly three lines, all of them import lines being widened. Lenient path: 1. `should accumulate a per-operator error when a Python operator's code generation fails` — the error is attributed to the right logical operator, carries the expected message, and compilation **continues** (the rest of the plan still lands in the physical plan and the storage set). 2. `should attribute each Python code-generation failure to its own logical operator` — two failing Python operators with two distinct messages, each keyed to its own id. 3. `should trim the marker's message and report it as a plain RuntimeException` 4. `should report no code-generation error for a well-formed Python operator` — asserts the operator really is still Python-based, so it cannot pass by quietly ceasing to be one. 5. `should not subject non-Python operators to the code-generation check` Strict path: 6. `in strict mode should throw immediately when a Python operator's code generation failed` 7. `in strict mode should not throw for a well-formed Python operator` Five of the seven drive the **real shipped `SortOpDesc`**, whose `generatePythonCode` opens with `require(attributes.nonEmpty, ...)` and a per-key `require` (`SortOpDesc.scala:34-38`). A Sort dropped on the canvas and left unconfigured is therefore a genuine, user-reachable route into the marker state, and it conveniently yields two distinct messages — which is what makes test 2's per-operator attribution meaningful. A configured Sort gives the negative control in tests 4 and 7: same operator, same code path, codegen simply succeeds. Test 3 needs a small test-only fixture (`PaddedFailurePyOp`, ~15 lines, modelled on `PythonOperatorDescriptorSpec`'s `ThrowingPyOp`): no shipped operator raises a whitespace-padded message, and without padding the `.trim` is unobservable. It is also the spec's only source-operator Python case. Two notes: - Four assertions hard-code `SortOpDesc`'s exact `require` wording, so rewording those messages in `workflow-operator` will fail these `workflow-compiler` tests. That cross-module coupling is deliberate — it pins the end-to-end string a user actually sees — but it is worth knowing. - Test 3 also asserts the reported message starts with `java.lang.RuntimeException: `. That prefix comes from `err.toString` at `WorkflowCompiler.scala:66` and is part of what the UI renders today; the assertion carries a comment saying so, so if that pre-existing wart is ever fixed there is one self-explaining test to update. ### Any related issues, documentation, discussions? Closes #7647 Builds on the specs added by #5019 / #5022 and the module unification in #6143. ### How was this PR tested? `sbt "WorkflowCompiler/testOnly *WorkflowCompilerSpec"` — 22 tests, all passing (15 pre-existing + 7 new). Running the whole module (`WorkflowCompiler/test`) is green too: 3 suites / 57 tests / 0 failures. `WorkflowCompiler/scalafmtCheck` and `WorkflowCompiler/Test/scalafmtCheck` are clean. Every new test was mutation-checked, twice and independently: the production check was temporarily broken, the suite re-run, and the file reverted (verified byte-identical afterwards). Highlights, with the tests that are the *sole* killer of a mutant: | Mutant in `WorkflowCompiler.scala` | Caught by | |---|---| | lenient arm appends **and then** throws (abort instead of continue) | 1 only | | record only the first codegen failure per compile | 2 only | | drop `.trim` on the captured message | 3 only | | skip the check for source operators (`&& !isSourceOperator`) | 3 only | | strict arm swallows instead of throwing (`case None => ()`) | 6 only | | flip the `errorList` arms (`Some` → throw, `None` → skip) | 1, 6 | | change the message prefix | 1, 2, 3, 6 | | `group(1)` → `group(0)` | 1, 2, 3, 6 | | marker regex changed so it never matches | 1, 2, 3, 6 | | regex relaxed to `(.*)` (matches every operator) | 1, 2, 3, 4, 6, 7 | | attribute the error to the wrong logical operator id | 1, 2, 3 | | build the error but never record it | 1, 2, 3, 6 | | `RuntimeException` → `IllegalArgumentException` | 3 | Two mutants survive, both semantically equivalent rather than gaps: reporting the *last* marker match instead of the first (a codegen-failure body is exactly the one-line marker, so first == last), and anchoring the regex with `^` — the latter is in fact the fix for a bug found along the way (below), and all 22 tests stay green under it, so the suite does not over-fit to the current unanchored check. One more mutant is worth calling out honestly: removing the `isPythonBased` guard entirely does not fail test 5 — it aborts the whole suite, because `PhysicalOp.getCode` throws an `IllegalAccessError` (a `LinkageError`, which is not `NonFatal`) for non-code operators. Test 5 therefore documents the guard's intent rather than detecting its removal. It is kept for that reason, with a self-check that stops it silently becoming a no-op. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) Co-authored-by: Meng Wang <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/31998823016 With regards, GitHub Actions via GitBox
