nevzheng opened a new issue, #14092: URL: https://github.com/apache/gravitino/issues/14092
### Describe the proposal **Goal.** A test's directory says what it needs, and one task name runs it, with no flags. **Why.** Six Gradle properties, an environment variable and a package path rule decide what a `test` task runs. `-PskipTests` runs integration tests only and passes green with zero tests in a module that has none; `-PskipDockerTests=false` with Docker stopped skips silently. Backend IT cells on `main` take 82 to 105 minutes ([run 39604804463](https://github.com/apache/gravitino/actions/runs/39604804463)), each with 28 `-x` exclusions, close to the 120-minute job timeout. **The recommendation.** - `test` holds hermetic tests: the only test task in `check`, and the presubmit gate. - `integrationTest` holds every test that needs a container or a Gravitino server; `deployIntegrationTest` runs the same classes against the built package. - Container-only tests are integration tests with a curated tag, so CI runs them once instead of once per backend matrix cell. - Inside a suite, organization lives in code: module and package, then a few registered tags, then JUnit `@Nested` and `@Suite`. No suite per external system or per feature. - CI shards are Gradle tasks computed from module declarations, with `others` as the complement. | Goal | Before | After | | --------------------------- | -------------------------------------------------------------------------- | ----------------------------------------------- | | Fast tests of a module | `./gradlew :m:test -PskipITs` | `./gradlew :m:test` | | Presubmit gate | `./gradlew build -PskipITs -PskipDockerTests=false` | `./gradlew check` | | Integration tests, embedded | `./gradlew :m:test -PskipTests -PtestMode=embedded -PskipDockerTests=false` | `./gradlew :m:integrationTest` | | Integration tests, deployed | `./gradlew compileDistribution`, then `-PtestMode=deploy` as above | `./gradlew :m:deployIntegrationTest` | | One CI backend shard | `test -PskipTests -PtestMode=<mode> -PjdbcBackend=<store>` and 28 `-x` | `./gradlew backendITHive -PjdbcBackend=<store>` | **Decisions asked** (comment by `<date>`; silence is lazy consensus): 1. `check` runs only `test` and static checks. 2. Two suites, not three: container-only tests are tagged integration tests. 3. One `integrationTest` suite per module; no per-system or per-feature suites. 4. Shards are declared in module build files and keep the `hive`, `lakehouse` and `others` shards of [#13495](https://github.com/apache/gravitino/pull/13495). **Sequencing.** Starts after the container framework stack ([#13900](https://github.com/apache/gravitino/issues/13900)) merges. [#13837](https://github.com/apache/gravitino/pull/13837) (draft) is superseded by this plan. ### Task list **Draft task list.** Stages and PR boundaries change as the SPIP is reviewed; each stage gets its own issue when it starts. Every stage PR shows the same executed test classes per CI job before and after, or lists the difference. - [ ] SPIP PR: `design-docs/testing/organization/SPIP-test-organization.md` - [ ] S1: `build-logic` seed with `registerVariant`, Docker value source, `DockerTestService`; core lanes run by task name with no Docker flag - [ ] S2: root `Test` defaults into `gravitino.java-conventions`; per-module `test` baseline - [ ] S3: `integrationTest` suite, `deployIntegrationTest`, tag registries, Docker fail fast; `test` keeps excluding the Docker tag and the path rule until S7 - [ ] S4: `integration-test-common` becomes a library; 39 consumers switch from `testArtifacts` - [ ] S5a: move JDBC catalog integration and container-only tests - [ ] S5b: move lakehouse catalog tests - [ ] S5c: move client tests - [ ] S5d: move Flink tests - [ ] S5e: move Spark tests - [ ] S5f: move Iceberg and Lance tests - [ ] S5g: move authorization, plugin and maintenance tests - [ ] S5h: move web and Trino tests - [ ] S6: CI by task name: `backendITHive`, `backendITLakehouse`, `backendITOthers`, one container-only job; zero `-x` - [ ] S7: delete the test flags, the path rule and `testArtifacts`; `client-python` uses its named tasks - [ ] S8a: test report and coverage aggregation - [ ] S8b: configuration cache on in CI - [ ] S9: docs sweep **Done when.** Zero test-selection flags in build files, workflows and user docs; no file under `src/test` outside core's backend lane classes imports `org.testcontainers`; backend IT runs shard task names with zero `-x`; the proposed `test` SLO (5 minutes per module, `check` under 20 minutes on CI) is tracked here against the S2 baseline. Design, options and rationale: SPIP (PR to follow). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
