The GitHub Actions job "Required Checks" on texera.git/gh-readonly-queue/main/pr-6238-a7989ee78e27ab7b909bdf8aaed586a5f030528c has failed. Run started by GitHub user Xiao-zhen-Liu (triggered by Xiao-zhen-Liu).
Head commit for run: cb5f2f5775edf8d70b8ceae2ecea1c3a05d5e9b5 / Neil Ketteringham <[email protected]> refactor: Refactor MockTexeraDB into a JVM singleton (#6238) ### What changes were proposed in this PR? Closes #6063 by completely refactoring the `MockTexeraDB` infrastructure to transition from a distributed process architecture to a centralized, shared-singleton architecture. 1. **High-Performance Shared Singleton Architecture:** Refactored the core setup by splitting the original `MockTexeraDB` trait into a companion `object MockTexeraDB` singleton and a lightweight `trait MockTexeraDB` interface wrapper. The background `EmbeddedPostgres` database engine process is now booted **exactly once** per JVM lifecycle via synchronized initialization. Individual test suites dynamically provision an isolated schema sandbox via cheap `CREATE DATABASE` queries instead of spawning heavy process-level instances. 2. **DDL Caching and Regex Optimization:** Removed massive runtime file I/O and processing bottlenecks. The full-text search index string regex modifications and SQL script cleaning are handled once and cached globally in memory (`ddlScript`), preventing parallel test suites from thrashing disk resources over the same `sql/texera_ddl.sql` file. 3. **Per-Suite HikariCP Connection Pooling:** Each isolated schema sandbox is now backed by a real HikariCP pool (bounded maximumPoolSize) rather than a single shared connection. This gives concurrent transactions distinct connections — matching production — so they no longer trample one connection's autoCommit flag. This pooling is the mechanism that lets the high-concurrency specs (e.g. DatasetResourceSpec) converge without deadlocks or connection-state leakage. 4. **Teardown API (shutdownDB → closeConnectionPool):** Because the trait now owns explicit pool resources, the teardown method was renamed to closeConnectionPool() and all in-tree callers updated. To preserve backward compatibility, shutdownDB() is retained as a thin alias (def shutdownDB(): Unit = closeConnectionPool()) so existing/incoming specs on main continue to compile unchanged. 5. **E2E specs moved onto embedded Postgres:** TestUtils.initiateTexeraDBForTestCases now provisions its isolated database on MockTexeraDB's embedded Postgres instead of an external test Postgres instance. This depends on #4179. The helper was also hardened — the DDL connection is now closed via Using.resource (no leak on failure) and a dead DROP DATABASE IF EXISTS on the freshly generated UUID name was removed. ### Performance Impact Moving database instantiation, parsing, and regex evaluation out of the distributed trait lifecycle and into a centralized static memory layer resulted in a dramatic reduction in test suite runtime. * **Main Branch Baseline / Prior Implementation:** 5 minutes 40 seconds * **This PR:** **4 minutes 5 seconds** (Total execution time reduced by ~28% locally). ### Any related issues, documentation, discussions? Follow up on #4525 where a refactor of `MockTexeraDB` was needed to allow for future concurrent execution of test suites. Structural baseline was cherry-picked and heavily adapted from #4527, specifically commit `bd93161` written by @Yicong-Huang. ### How was this PR tested? Verified by running full, clean compilations and local stress tests on JDK 17 to validate that the structural split preserves total API backward compatibility across the codebase. Additionally validated that high-concurrency race conditions (such as the asynchronous multi-threaded tests inside `DatasetResourceSpec`) converge gracefully without experiencing deadlocks, thread crashes, or connection state leakage. All 1,600+ test suite assertions across the system are green. Run configuration command used to validate the project modules: ```bash sbt clean compile test ``` ### Was this PR authored or co-authored using generative AI tooling? Yes. Generative AI (Gemini) was utilized as an external assistant to analyze complex multi-threaded JDBC/jOOQ connection trace errors and help structure the safe connection lifecycle fallback logic in SqlServer.scala. No direct IDE-integrated automation tools were used to auto-generate codebase files. --------- Signed-off-by: Neil Ketteringham <[email protected]> Co-authored-by: Yicong Huang <[email protected]> Co-authored-by: Copilot Autofix powered by AI <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/30302584914 With regards, GitHub Actions via GitBox
