jprinet opened a new pull request, #45217: URL: https://github.com/apache/superset/pull/45217
### SUMMARY Adds [Develocity](https://develocity.apache.org) integration for the Python, npm and Docker builds, using the [gradle/develocity-actions](https://github.com/gradle/develocity-actions) v2.5 setup actions. Read more on [Develocity](https://develocity.ai/). CI builds publish Build Scans to [develocity.apache.org](https://develocity.apache.org). Nothing changes for local builds, and no Develocity package is added to the project's dependencies. **What to expect.** Each CI invocation of pytest, npm or `docker buildx build` produces a Build Scan. - **Test results:** pytest and Jest results per test, with durations, outcomes and failure output. Each test can be followed across runs, which shows flaky and slow tests. - **Failures and performance:** each Build Scan holds the console log, environment and timeline. Develocity groups recurring failures and tracks durations across builds. - **Docker:** each image build shows per-step timing and which layers came from the cache. npm Build Scans add dependency resolution and download details. **Python** (`gradle/develocity-actions/setup-python`, agent 0.10.1) - `setup-backend` runs `setup-python`, which installs the agent into the job's interpreter. The key is passed to the jobs that run pytest or another tool the agent records. - `.develocity.py` at the root and in `superset-extensions-cli` set the project name (`superset`), the `CI`/`Local` tag and the custom values. The agent expects one next to each `pyproject.toml` it runs under. **npm** (`gradle/develocity-actions/setup-npm`, agent 4.1.1) - On the runner (`superset-websocket`, `embedded-sdk-test`, `superset-translations`, `tech-debt`, `superset-e2e`), `setup-npm` runs after `setup-node`. It installs the agent outside the project and wraps `npm` and `npx`. It also adds the Jest reporter, so the project needs no configuration file and no `jest.config.js` change. - In the `superset-node-ci` containers (`superset-frontend.yml`, `chromatic.yml`), `setup-npm` installs the agent on the runner, and `docker run` mounts it with the `DEVELOCITY_*` settings. This uses the same reporter injection flag the action's own wrappers set. - The `test` script in `superset-frontend/package.json` keeps the inherited `NODE_OPTIONS` (`NODE_OPTIONS="$NODE_OPTIONS ..."`). Otherwise Jest would run without the agent. On Windows, `cross-env` expands an unset `$NODE_OPTIONS` to a literal `%NODE_OPTIONS%`. - Project names come from each `package.json`: `superset`, `superset-websocket` and `@superset-ui/embedded-sdk`. **Docker** (`gradle/develocity-actions/setup-docker`, agent 0.9.0) - One step in `docker-build`, `frontend-build` and `chromatic`. The action stops the agent at the end of the job, waits for every scan and lists them in the job summary. Package scanning is off. **Grouping the Build Scans of a push.** Every Build Scan carries a `CI_commit_sha` custom value. Search on it to find all the Build Scans a push produced. All Develocity steps run only when `DEVELOCITY_ACCESS_KEY` is set, so fork pull requests and scheduled runs are unaffected. Not covered: - **Scheduled, release and nightly workflows:** they aren't triggered by a push. - **`pre-commit` npm hooks** and **`github-action-validator`**. - **npm steps inside the Dockerfile:** they appear in the Docker Build Scan, not as npm Build Scans. - **Test results for Vitest (`superset-websocket`, `superset-embedded-sdk`) and Playwright:** neither has a Develocity reporter. Their npm runs are still recorded. - **The `docker compose` build:** it doesn't pass the `CI_commit_sha` build argument. ### DEVELOCITY SCREENSHOTS All Build Scans of one push, found with a single `CI_commit_sha` search. pytest (Python), Jest (npm) and Docker image builds sit side by side under the `superset` project:  Every Build Scan carries the same `CI_commit_sha`, along with the workflow, job and run it came from:  ### TESTING INSTRUCTIONS Before merging: - Request a develocity.apache.org access key from ASF Infra (INFRA Jira ticket) - The actions must be on the ASF allow list at v2.5 (apache/infrastructure-actions#1383). Until then, `asf-allowlist-check` fails. ### ADDITIONAL INFORMATION - [x] Has associated issue: discussion #45216 - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API Generated-by: Claude Code (Claude Opus 5.5), following the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html). It set up the gradle/develocity-actions steps, edited the workflows, ran the checks above and wrote this description. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
