This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25203-bb21f51013a3f476c5e510de745c77e194fcb584 in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 615e9b42250d094dbc44fc0dd21b4e13f876171d Author: Kumar Ujjawal <[email protected]> AuthorDate: Sat Sep 12 11:31:52 2026 +0000 ci: require extended tests in the merge queue (#25203) ## Which issue does this PR close? - Closes #21241. ## Rationale for this change Extended-test failures can reach `main` because the checks are optional on PRs and run after merge. Require all three suites in the merge queue so failures block merging without running the suites on every PR update. ## What changes are included in this PR? - Add an unfiltered `pull_request` trigger and a `merge_group` trigger. Skip the three jobs on PR updates so their required checks allow queue entry. - Add the Rust extended tests, forced hash-collision tests, and SQLite SQL logic tests to the required checks in `.asf.yaml`. - Remove duplicate runs on pushes to `main`. Preserve release-branch pushes and manual dispatch, including the optional PR head SHA. - Update the contributor testing guide. Test commands, feature flags, runners, concurrency settings, and benchmark checks are unchanged. ## What is the testing strategy for this PR? This change only affects CI scheduling and documentation, so it adds no Rust tests. - The required-status-check validator passes for all 35 configured checks. - YAML comparison confirms that the three jobs are unchanged except for their conditions. PR and merge-group triggers have no path filters. - Documentation formatting, workflow tool-install checks, and `git diff --check` pass. - Actionlint 1.7.12 reports the same unsupported `parallel` blocks on the base and this branch. Both workflows pass when those existing blocks are flattened in temporary lint inputs. The [initial hosted PR run](https://github.com/apache/datafusion/actions/runs/34630744756) reports all three extended jobs as skipped. The hosted required-status-check validator also passes. Merge-group execution and enforcement still need validation when the PR enters the queue. ## Are there any user-facing changes? Contributors wait for extended tests in the merge queue instead of on PR updates or after merge. There are no query behavior or public API changes. --- .asf.yaml | 3 +++ .github/workflows/extended.yml | 41 +++++++++++++------------------- docs/source/contributor-guide/testing.md | 18 +++++++------- 3 files changed, 29 insertions(+), 33 deletions(-) diff --git a/.asf.yaml b/.asf.yaml index 34d036e19b..f235c6dd97 100644 --- a/.asf.yaml +++ b/.asf.yaml @@ -68,6 +68,8 @@ github: - "cargo check datafusion-functions features" - "cargo check datafusion-spark features" - "cargo test (amd64)" + - "cargo test 'extended_tests' (amd64)" + - "cargo test hash collisions (amd64)" - "cargo test datafusion-cli (amd64)" - "cargo examples (amd64)" - "cargo test doc (amd64)" @@ -76,6 +78,7 @@ github: - "verify benchmark results (amd64)" - "Run sqllogictest with Postgres runner" - "Run sqllogictest in Substrait round-trip mode" + - "Run sqllogictests with the sqlite test suite" - "cargo test (macos-aarch64)" - "Verify Vendored Code" - "Check cargo fmt" diff --git a/.github/workflows/extended.yml b/.github/workflows/extended.yml index d2f1f71175..cdddc3c9a4 100644 --- a/.github/workflows/extended.yml +++ b/.github/workflows/extended.yml @@ -21,31 +21,18 @@ concurrency: group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }} cancel-in-progress: true -# https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#running-your-pull_request-workflow-when-a-pull-request-merges -# -# These jobs are not run as part of PR checks as they are time-consuming -# and should not fail often. -# -# We still run them as they provide important coverage to ensure correctness -# in the (very rare) event of a hash failure or sqlite library query failure. +# Run these required checks in the merge queue before merging into main. +# On PR updates, skip the jobs so their required checks allow queue entry +# without running the expensive suites. Do not add pull_request path filters: +# a skipped workflow would leave required checks pending. on: + pull_request: + merge_group: push: branches: - - main - # support extended test suite for release candidate branches, - # it is not expected to have many changes in these branches, - # so running extended tests is not a burden + # Release branches retain post-push coverage. main is covered by the + # merge queue, so do not repeat the tests after merging. - 'branch-*' - # Also run for changes to some critical areas that are most likely - # to trigger errors in extended tests - pull_request: - branches: [ '**' ] - paths: - - 'datafusion/physical*/**/*.rs' - - 'datafusion/expr*/**/*.rs' - - 'datafusion/optimizer/**/*.rs' - - 'datafusion/sql/**/*.rs' - - 'datafusion-testing' workflow_dispatch: inputs: pr_head_sha: @@ -60,13 +47,15 @@ jobs: # Run extended tests (with feature 'extended_tests') linux-test-extended: name: cargo test 'extended_tests' (amd64) + # Long-running job: run in the merge queue, on release pushes, or manually. + if: github.event_name != 'pull_request' runs-on: ${{ vars.USE_RUNS_ON == 'true' && format('runs-on={0},family=m8a+m7a+c8a,cpu=32,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }} # note: do not use amd/rust container to preserve disk space steps: - uses: runs-on/action@efac073ea2507ec18797de3a81704201ade11d9d # v2.3.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push + ref: ${{ github.event.inputs.pr_head_sha }} # otherwise use the event's commit, including the merge group submodules: true fetch-depth: 1 - name: Free Disk Space (Ubuntu) @@ -111,6 +100,8 @@ jobs: # Check answers are correct when hash values collide hash-collisions: name: cargo test hash collisions (amd64) + # Long-running job: run in the merge queue, on release pushes, or manually. + if: github.event_name != 'pull_request' runs-on: ${{ vars.USE_RUNS_ON == 'true' && format('runs-on={0},family=m8a+m7a+c8a,cpu=16,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }} container: image: amd64/rust @@ -118,7 +109,7 @@ jobs: - uses: runs-on/action@efac073ea2507ec18797de3a81704201ade11d9d # v2.3.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push + ref: ${{ github.event.inputs.pr_head_sha }} # otherwise use the event's commit, including the merge group submodules: true fetch-depth: 1 - name: Setup Rust toolchain @@ -133,6 +124,8 @@ jobs: sqllogictest-sqlite: name: "Run sqllogictests with the sqlite test suite" + # Long-running job: run in the merge queue, on release pushes, or manually. + if: github.event_name != 'pull_request' runs-on: ${{ vars.USE_RUNS_ON == 'true' && format('runs-on={0},family=m8a+m7a+c8a,cpu=32,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }} container: image: amd64/rust @@ -141,7 +134,7 @@ jobs: - parallel: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push + ref: ${{ github.event.inputs.pr_head_sha }} # otherwise use the event's commit, including the merge group submodules: true fetch-depth: 1 # Don't use setup-builder to avoid configuring RUST_BACKTRACE which is expensive diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.md index c344d67159..44bd327013 100644 --- a/docs/source/contributor-guide/testing.md +++ b/docs/source/contributor-guide/testing.md @@ -111,7 +111,7 @@ cargo test --profile=ci --test sqllogictests -- --complete Like similar systems such as [DuckDB](https://duckdb.org/dev/testing), DataFusion has chosen to trade off a slightly higher barrier to contribution for longer term maintainability. -DataFusion has integrated [sqlite's test suite](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) as a supplemental test suite that is run whenever a PR is merged into DataFusion. To run it manually please refer to the [README](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite) file for instructions. +DataFusion runs [sqlite's test suite](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) in the merge queue before merging PRs into `main`. For local instructions, see [Running Tests: sqlite](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite). ## Snapshot testing (`cargo insta`) @@ -127,15 +127,15 @@ cargo insta review ## Extended Tests -In addition to the standard CI test suite that is run on all PRs prior to merge, -DataFusion has "extended" tests (defined in [extended.yml]) that are run on each -commit to `main`. These tests rarely fail but take significantly longer to run -than the standard test suite and add important test coverage such as ensuring -correctness when there are hash collisions and running the relevant portions of -the entire [sqlite test suite]. You can run the extended tests -locally by following the [instructions in the documentation]. +DataFusion has extended tests (defined in [extended.yml]) that take significantly +longer to run than the standard suite. They provide additional correctness +coverage and must pass in the merge queue before a PR merges into `main`. + +To conserve CI resources, these tests do not run on ordinary PR updates. They +also run on pushes to release branches (`branch-*`). You can run them manually. + +For local SQLite test instructions, see the [instructions in the documentation]. -[sqlite test suite]: https://www.sqlite.org/sqllogictest/dir?ci=tip [instructions in the documentation]: https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest#running-tests-sqlite [extended.yml]: https://github.com/apache/datafusion/blob/main/.github/workflows/extended.yml --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
