aglinxinyuan opened a new pull request, #6884: URL: https://github.com/apache/texera/pull/6884
### What changes were proposed in this PR? `LoopIntegrationSpec` costs **~4m16s of the ~6-minute `amber-integration` test step (~70%)** on every PR. The cost tracks worker spawns almost perfectly (~2s per Python worker boot + Flight handshake; 132 spawns across the suite), because every loop iteration respawns each worker in the re-executed regions: | Test | Time | Worker spawns (py + jvm) | |---|---:|---:| | single loop (Python-only) | 17s | 6 + 1 | | nested 3×3 (Python-only) | 63s | 24 + 1 | | single + Limit | 19s | 6 + 4 | | nested + Limit | 64s | 24 + 10 | | nested + chain (Limit→Sleep) | 72s | 24 + 19 | | single + chain | 21s | 6 + 7 | Two cases are **strict subsets** of the nested-chain case, which already covers nested routing, counter magnitude across JVM hops, the JVM→JVM state handoff, and both operator kinds in one workflow: - `nested + Limit` (~64s) — nested × Limit ⊂ nested × (Limit → Sleep) - `single + chain` (~21s) — chain ⊂ nested chain This PR removes those two, saving **~85s (~33% of the loop suite)** with no unique coverage lost. Retained matrix: Python-only single (base), Python-only nested (pins the no-JVM path), single + Limit (verbatim repro of the #6660 bug), nested + chain (superset JVM case). The subsumption reasoning is recorded in the retained test's comment so the cases don't get re-added. The deeper fix (reusing workers across loop iterations instead of respawn-per-jump) is an engine design change and out of scope here. ### Any related issues, documentation, discussions? Follow-up to #6661 (which introduced the JVM loop-body cases). Related engine context: #6660. ### How was this PR tested? Test-only change (removes two e2e cases; no source touched). `WorkflowExecutionService/Test/compile` + `scalafmtCheckAll` + `scalafixAll --check` pass locally (Java 17); the remaining suite runs in the `amber-integration` CI job on this PR. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Fable 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
