nevzheng opened a new pull request, #13838:
URL: https://github.com/apache/gravitino/pull/13838
### What changes were proposed in this pull request?
The core MySQL and PostgreSQL test tasks can use a database the environment
provides instead of starting their own Docker container. Injected dependencies
win; Docker is the fallback.
- Set `GRAVITINO_TEST_MYSQL_URL` (an admin JDBC URL) with `_USER` and
`_PASSWORD`, or the `GRAVITINO_TEST_POSTGRESQL_*` equivalents.
- With them set, `coreMySQLTest` / `corePostgreSQLTest` start no container,
skip the "requires Docker" check, and join `check` / `build`. Each test fixture
still gets its own new database on that server.
- The classes that started containers themselves
(`AbstractEntityStorageTest` and the nested
`TestJdbcPartitionStatisticStorageIT` MySQL/PostgreSQL tests) now go through
the same provisioner, so a run with an injected database starts no container at
all.
- `BackendTestExtension` uses the provisioner's credentials instead of
assuming `root`/`root`.
- Without the variables, nothing changes: the shared build-wide container
from #13553, or per-test containers outside Gradle.
This PR is the last commit on the stack; the earlier commits belong to
#13517, #13518, #13530, #13553 and #13837.
### Why are the changes needed?
The database tests depend on Testcontainers, which needs a Docker daemon the
test JVM can reach. That is a poor fit for many CI setups:
- **Kubernetes runners** usually have no Docker daemon. Docker-in-Docker
needs privileged pods, and mounting the node's socket is a security risk many
clusters forbid.
- **Managed or pre-provisioned databases** (service containers, a shared
test server, a cloud instance) cannot be used today, even when they are already
running next to the job.
- **Start-up cost and image pulls** are paid on every run, and rate limits
or registry outages fail the build for reasons unrelated to the code.
Injecting the database through the environment keeps Testcontainers as the
default for laptops and the current GitHub Actions jobs. It also lets anyone
deploying these tests on Kubernetes run them as a pod with a MySQL or
PostgreSQL sidecar or service.
Related to: #13516
### Does this PR introduce _any_ user-facing change?
No production changes. Test-only:
- New optional environment variables: `GRAVITINO_TEST_MYSQL_URL` / `_USER` /
`_PASSWORD` and `GRAVITINO_TEST_POSTGRESQL_URL` / `_USER` / `_PASSWORD`.
- The user must be allowed to create databases. Databases created on an
injected server are not dropped afterwards, so use a disposable server.
- Credentials are passed to the test JVM at execution time, so they stay out
of Gradle's task inputs and cache key.
### How was this patch tested?
Locally (macOS, JDK 17, Gradle 8.2). MySQL 8.0 and PostgreSQL 13 were
started with `docker run` outside Gradle, on non-default ports with non-default
credentials, to stand in for injected databases:
- `GRAVITINO_TEST_*` set, `./gradlew :core:corePostgreSQLTest
:core:coreMySQLTest -PskipDockerTests=true`:
- PostgreSQL: 588 tests, 0 failures, 3 skipped.
- MySQL: 588 tests, 0 failures, 2 skipped.
- No new container was created during the run (`docker ps -a` before and
after).
- The injected servers held the databases the tests created.
- The first attempt failed on hard-coded `root`/`root` credentials in
`BackendTestExtension`; fixed here.
- Without the variables, `-PskipDockerTests=false`: `TestEntityStorage`,
`TestJdbcPartitionStatisticStorageIT$MySQLTest` / `$PostgreSQLTest`,
`TestJobMetaService` and the rest of the selected classes pass on the shared
container as before.
- `./gradlew :core:spotlessCheck :integration-test-common:spotlessCheck
:core:compileTestJava`
Not covered: no CI job sets the variables yet, so this path is exercised
only locally.
**Stack:** #13517 → #13518 → #13530 → #13553 → #13837 → **this PR**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
