aglinxinyuan opened a new pull request, #8940: URL: https://github.com/apache/texera/pull/8940
### What changes were proposed in this PR? `TestUtils.runWorkflowAndReadResults` now fails as soon as a worker reports an error. Before, an operator that threw inside an e2e or integration test made the test wait out its whole deadline and fail with `com.twitter.util.TimeoutException`, without the operator's error. `withRetry` then ran it a second time. A worker whose executor throws does not raise a `FatalError`. It sends the error to the coordinator as an ERROR `ConsoleMessage` and pauses itself: `DataProcessor.handleExecutorException` on the JVM, `Context.report_exception` in pyamber. The coordinator forwards the message to the client. No state update follows, so the run never reaches COMPLETED. The harness listened only for COMPLETED and `FatalError`. The harness now also listens for `ConsoleMessage`. On an ERROR message it fails the run with a `WorkflowRuntimeException` that carries the message's title, its stack trace and the failing worker. That is the same exception type a `FatalError` raises when an executor fails to initialize. Print, command and debugger messages are ignored. `SyncExecutionResource` already ends a run on an ERROR console message in the same way. ``` Before: operator throws -> ERROR console message, worker paused -> harness waits out the deadline -> TimeoutException After: operator throws -> ERROR console message, worker paused -> harness fails at once with the operator's error ``` ### Any related issues, documentation, discussions? Closes #8939. ### How was this PR tested? Tests first: the first commit adds the tests and fails on `main`, and the second commit makes them pass. New `TestUtilsSpec`: | Case | Pinned | |---|---| | CSV scan -> Type Casting of the text column `Region` to integer | The harness fails with a `WorkflowRuntimeException` that names the parse error and the Type Casting worker. On `main` it fails with `TimeoutException` after 1 min 3 s; with this PR it fails in under 4 s. | | The same workflow, casting `Units Sold` (whole numbers) | The harness returns all 100 rows, typed integer. | | `workerError` on an ERROR console message | The error carries the title, the stack trace and the worker id. An empty stack trace adds no trailing newline. | | `workerError` on print, command, debugger and unrecognized messages | No error. | A Python UDF that raises was checked once with a temporary spec that is not part of this PR. The harness failed after 9.9 s with `ValueError: python udf failed on purpose` and the Python traceback, naming the UDF worker. The specs that use the harness pass, all on the first attempt: ``` STORAGE_ICEBERG_CATALOG_TYPE=postgres sbt "WorkflowExecutionService/testOnly *TestUtilsSpec *DataProcessingSpec *PauseSpec *ReconfigurationSpec" AMBER_TEST_FILTER=integration-only STORAGE_ICEBERG_CATALOG_TYPE=postgres sbt "WorkflowExecutionService/testOnly *LoopIntegrationSpec *MultiRegionWorkflowIntegrationSpec *ReconfigurationIntegrationSpec" sbt "WorkflowExecutionService/scalafmtCheckAll" "WorkflowExecutionService/scalafixAll --check" ``` ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
