sandynz commented on PR #38274:
URL: https://github.com/apache/shardingsphere/pull/38274#issuecomment-3977023239
The e2e-sql workflow ran the full 119-job static matrix on every PR
regardless of what changed. Replace it with a `dorny/paths-filter`-driven
dynamic matrix that runs only the jobs relevant to the PR's changed files,
reducing CI load by 50–90% for most PRs.
## Changes
- **`.github/workflows/resources/filter/e2e-sql-filters.yml`** — 18 boolean
path-filter labels across four dimensions:
- Adapter: `adapter_proxy`, `adapter_jdbc`
- Mode: `mode_standalone`, `mode_cluster`, `mode_core`
- Database: `database_mysql`, `database_postgresql`
- Feature/scenario: `feature_sharding`, `feature_encrypt`,
`feature_readwrite_splitting`, `feature_shadow`, `feature_mask`,
`feature_broadcast`, `feature_distsql`, `feature_sql_federation`
- Full-fallback triggers: `core_infra`, `test_framework`, `pom_changes`
- **`.github/workflows/resources/scripts/generate-e2e-sql-matrix.sh`** —
reads the 18 filter outputs and emits a compact JSON matrix to `$GITHUB_OUTPUT`
with three outputs: `matrix`, `has-jobs`, `need-proxy-image`. Logic:
- `core_infra | test_framework | pom_changes = true` → emit full 119-job
matrix unchanged
- No relevant label changed → `has-jobs=false`, skip all downstream jobs
- Otherwise: compute adapters/modes/databases from dimension labels;
resolve scenarios from feature labels (smoke set `empty_rules, db, tbl,
encrypt, readwrite_splitting, passthrough` when no feature label fires); apply
all original 5 exclude rules and the MySQL 8.3.0 extra passthrough include
- **`.github/workflows/e2e-sql.yml`** — replaces the static `matrix:` block
with four jobs:
1. `global-environment` — unchanged
2. `detect-and-generate-matrix` — runs filter action + script, exposes
`matrix`/`has-jobs`/`need-proxy-image`
3. `build-e2e-image` — conditioned on `has-jobs == 'true' &&
need-proxy-image == 'true'`
4. `e2e-sql` — uses
`fromJSON(needs.detect-and-generate-matrix.outputs.matrix)` with `always()`
guard so it handles the skipped build-image case cleanly
Example output for a PR touching only `features/sharding/`:
```json
{"include":[
{"adapter":"proxy","mode":"Standalone","database":"MySQL","scenario":"db","additional-options":""},
{"adapter":"proxy","mode":"Cluster","database":"MySQL","scenario":"sharding_and_encrypt","additional-options":""},
...
]}
```
60 jobs instead of 119.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
