MaxGekk opened a new pull request, #59611: URL: https://github.com/apache/spark/pull/59611
### What changes were proposed in this pull request? This PR raises the `PROCESSES` parameter of the Oracle Free container used by the JDBC docker integration tests from the image default of 200 to 1000. `PROCESSES` is a static parameter, so it has to be in the spfile before the instance starts. `OracleDatabaseOnDocker` now starts the container through a small wrapper entrypoint, `oracle-docker-entrypoint.sh`. The script starts the instance in NOMOUNT, runs `ALTER SYSTEM SET PROCESSES = 1000 SCOPE = SPFILE`, aborts the instance and then `exec`s the image's own `container-entrypoint.sh`, which starts the listener and the database as before. The script is copied to a temporary directory and bind-mounted read-only into the container from `beforeContainerStart`, the same way the Kerberos suites already mount their setup scripts. Every Oracle suite picks this up: the v1 and v2 `OracleIntegrationSuite`, `OracleNamespaceSuite` and `OracleJoinPushdownIntegrationSuite`. The wrapper needs a gvenzl/oracle-free style image that provides `container-entrypoint.sh`, like the default `sarutak/oracle-free` image, so it is applied only when the image name contains `oracle-free`. Other images, such as the `oracle/database` images built from `oracle/docker-images` that the suites' scaladoc mentions, start through their own entrypoint as before. The change happens before the listener exists, so the harness can never connect to the database and then lose that connection to a restart. A script in `/container-entrypoint-startdb.d` would have that race, because those scripts run after the database is already open and the passwords are already set. ### Why are the changes needed? `org.apache.spark.sql.jdbc.v2.OracleIntegrationSuite` fails intermittently, about a minute into the suite, in the `scan with aggregate push-down: ... without DISTINCT` tests. The error is `java.sql.SQLException: ORA-12516: Cannot connect to database. Listener ... does not have a protocol handler for TCP ready or registered for service freepdb1`. The failures come in one or two bursts of 1 to 8 tests. Each burst lasts under a second, and when there are two they are about 3 seconds apart. After that the suite recovers without any intervention. It is a known flake in PR CI runs, usually re-run (for example in #57809 and #57551). The `... with DISTINCT` variants of these tests are excluded for Oracle, so the failing block is always the `without DISTINCT` one. The instance never runs out of processes. During a full run of the suite, `v$resource_limit` shows `processes` at about 88 current and 92 peak out of 200, so no connections are leaking. The limit that is hit is the listener's own estimate of the load. The listener refuses a dedicated connection once the process count the instance last reported to it, plus every connection it has handed off since, reaches `PROCESSES`. The instance (LREG) sends that report only every 3 seconds or more, and less often when the load changes little relative to the limit. I measured this directly against the test image (`sarutak/oracle-free:23.26.2-slim`): with 88 processes reported, the first refusal came after exactly 112 accepted connections since the last `service_update`, and 88 + 112 = 200. The handler stayed `state:blocked` until the next report, about 2 seconds later. The v2 suite opens about 770 short-lived connections in about 40 seconds. Each query with `numPartitions = 4` opens a connection per partition, on top of the driver-side metadata connections. In the listener log of a local run, the busiest 3-second window had 99 connections, against the 112 that are available. A slightly denser burst on a CI runner, or a report that arrives a little late, crosses the line. Raising the instance's startup timeout (SPARK-57603) does not help, because the database is already up when this happens. With `PROCESSES = 1000`, the same 8-, 16- and 32-wide bursts that were refused before are all accepted, and the listener never leaves `state:ready`. The fixed suite's busiest listener window is about 220 connections, against roughly 910 available. ### Does this PR introduce _any_ user-facing change? No. Only the test harness changes. ### How was this patch tested? Locally, with Docker and `ENABLE_DOCKER_INTEGRATION_TESTS=1`: - `build/sbt -Pdocker-integration-tests "docker-integration-tests/testOnly org.apache.spark.sql.jdbc.v2.OracleIntegrationSuite"` passed in 3 of 3 runs: 63 tests succeeded, 22 ignored. During each run I polled `v$resource_limit` and `lsnrctl services` inside the container. The limit was 1000, `processes` peaked at 90 to 91, the listener refused 0 connections, and it was never blocked. - `docker-integration-tests/testOnly org.apache.spark.sql.jdbc.OracleIntegrationSuite org.apache.spark.sql.jdbc.v2.OracleNamespaceSuite org.apache.spark.sql.jdbc.v2.join.OracleJoinPushdownIntegrationSuite` passed: 55 tests succeeded, 5 ignored. - The flake did not reproduce locally without the fix, including with the container capped to 1 CPU. The evidence for the cause is the listener measurements above. - After restricting the wrapper to `oracle-free` images, the v2 `OracleIntegrationSuite` passed again (63 succeeded, 22 ignored). - `docker-integration-tests/scalastyle` and `docker-integration-tests/Test/scalastyle` pass. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5 and Claude Fable 5.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
