Jackie-Jiang opened a new pull request, #20202: URL: https://github.com/apache/pinot/pull/20202
The batch quickstart starts multiple servers in one JVM, where `DimensionTableDataManager` is a singleton keyed only by table name. Concurrent server initialization overwrites the shared manager's locks, directories, and throttlers while segments are loading. This caused semaphore overflow and failed dimension joins in the [quickstart CI job](https://github.com/apache/pinot/actions/runs/42606091575/job/129122358911). Run the batch quickstart with two servers in separate JVMs, preserving distributed query coverage while isolating server state. Add an opt-in process launcher to `QuickstartRunner` that transfers effective configuration and relevant JVM settings, bounds each server heap to 128 MiB initial / 1 GiB maximum, and separates logs and temporary files. Require a child startup marker and HTTP readiness, retain startup diagnostics, and clean up children on partial startup failure, shutdown, or loss of the parent. Validation: - Scoped JDK 25 reactor build and 15 focused tests passed, including six real-child-JVM tests for readiness, unrelated listeners, early exit, timeout cleanup, parent-pipe closure, and configuration roundtrip: ```sh ./mvnw -pl pinot-tools -am \ '-Dtest=QuickstartServerProcessTest,BaseQuickStartTest,QuickStartTest' \ -Dsurefire.failIfNoSpecifiedTests=false \ -Dmaven.compiler.showDeprecation=true -Dmaven.compiler.showWarnings=true package ``` - Live two-table BATCH smoke test: two distinct server PIDs, dimension segment ONLINE on both servers, baseball count 97,889, and successful multi-stage and `lookup()` joins on three executions each. - Normal shutdown and an injected abrupt exit of the second server both left no child processes and removed quickstart data; startup-failure diagnostics were retained. - `spotless:apply`, `checkstyle:check`, `license:format`, and `license:check` passed for `pinot-tools`; root license validation passed for the CI script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
