This is an automated email from the ASF dual-hosted git repository.
Yicong-Huang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new f59dde5aa1 perf(ci): use uv pip for faster Python dep install (#4521)
f59dde5aa1 is described below
commit f59dde5aa1ac029b2d1ceb49d4af7d4d84ae698a
Author: Yicong Huang <[email protected]>
AuthorDate: Sat Apr 25 22:22:14 2026 -0700
perf(ci): use uv pip for faster Python dep install (#4521)
### What changes were proposed in this PR?
Both the `scala` and `python` jobs in
`.github/workflows/github-action-build.yml` install Python dependencies
via `pip install -r requirements.txt`. That step alone takes:
- `scala` job: **~1m 21s**
- `python` matrix: **1m 12s – 4m 41s** (3.13 is slowest because some
packages lack prebuilt wheels for it)
An earlier attempt on this PR turned on the built-in pip wheel cache
(`actions/setup-python` + `cache: 'pip'`) and saw only ~14s saved — the
bottleneck is install (resolve + extract + write site-packages for ~230
packages), not download.
This PR switches both jobs to `uv pip install --system`. uv is a Rust
reimplementation of pip with no transitive deps, so installing it via
`python -m pip install uv` adds only ~3s, and the same wheel set then
installs in ~10s instead of ~70s on the same runner.
No new third-party GitHub Action is added — uv is fetched as a regular
pip package — so this stays within the ASF Infra GitHub Actions
allowlist already used by this repo (`sbt/setup-sbt`,
`coursier/cache-action`, `docker/*`, `amannn/*`, `apache/*`).
### Any related issues, documentation, discussions?
Closes #4519. Companion to #4508 (which combined the two lint sbt
invocations).
### How was this PR tested?
CI on this PR — comparing `Install dependencies` step time before vs
after on both `scala` and `python (3.10|3.11|3.12|3.13)` jobs. Earlier
exploratory commits on this branch tried caching
`target/scala-2.13/{classes,zinc,src_managed}` directly; that broke
`scalafix` because zinc skipped the compile that produces the SemanticDB
files scalafix needs, so the sbt-target cache was dropped.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.7)
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
---
.github/workflows/github-action-build.yml | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/.github/workflows/github-action-build.yml
b/.github/workflows/github-action-build.yml
index b385c878e7..a8a333ecb3 100644
--- a/.github/workflows/github-action-build.yml
+++ b/.github/workflows/github-action-build.yml
@@ -119,9 +119,9 @@ jobs:
run: python --version || python3 --version
- name: Install dependencies
run: |
- python -m pip install --upgrade pip
- if [ -f amber/requirements.txt ]; then pip install -r
amber/requirements.txt; fi
- if [ -f amber/operator-requirements.txt ]; then pip install -r
amber/operator-requirements.txt; fi
+ python -m pip install uv
+ if [ -f amber/requirements.txt ]; then uv pip install --system -r
amber/requirements.txt; fi
+ if [ -f amber/operator-requirements.txt ]; then uv pip install
--system -r amber/operator-requirements.txt; fi
- name: Setup sbt launcher
uses: sbt/setup-sbt@508b753e53cb6095967669e0911487d2b9bc9f41 # v1.1.22
- uses: coursier/cache-action@4e2615869d13561d626ed48655e1a39e5b192b3c #
v6.4.9
@@ -162,9 +162,9 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
- python -m pip install --upgrade pip
- if [ -f amber/requirements.txt ]; then pip install -r
amber/requirements.txt; fi
- if [ -f amber/operator-requirements.txt ]; then pip install -r
amber/operator-requirements.txt; fi
+ python -m pip install uv
+ if [ -f amber/requirements.txt ]; then uv pip install --system -r
amber/requirements.txt; fi
+ if [ -f amber/operator-requirements.txt ]; then uv pip install
--system -r amber/operator-requirements.txt; fi
- name: Install PostgreSQL
run: sudo apt-get update && sudo apt-get install -y postgresql
- name: Start PostgreSQL Service
