This is an automated email from the ASF dual-hosted git repository. tqchen pushed a commit to branch unified-jenkins-900 in repository https://gitbox.apache.org/repos/asf/tvm-tirx.git
commit 7e2f24b4f3dc6f13ec50ebfff81b4579dba72089 Author: tqchen <[email protected]> AuthorDate: Sun Nov 1 00:55:49 2026 +0000 [CI] Unify Jenkins builds around manylinux wheel artifacts --- AGENTS.md | 7 +- ci/jenkins/README.md | 45 +++++ ci/jenkins/arm_jenkinsfile.groovy | 46 ------ ci/jenkins/common.groovy | 40 ++--- ci/jenkins/cpu_jenkinsfile.groovy | 66 -------- ci/jenkins/docker-images.ini | 8 +- ci/jenkins/docker/bash.sh | 34 ++-- ci/jenkins/gpu_jenkinsfile.groovy | 147 ----------------- ci/jenkins/jenkinsfile.groovy | 199 +++++++++++++++++++++++ ci/jenkins/scripts/determine_docker_images.py | 4 +- tests/scripts/task_build_gpu.sh | 7 + tests/scripts/task_build_gpu_other.sh | 20 +-- tests/scripts/task_install_build_dependencies.sh | 8 + tests/scripts/task_python_docs.sh | 2 + tests/scripts/test_task_install_dist_wheels.sh | 8 + 15 files changed, 332 insertions(+), 309 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 3c16928b0..ec437bb94 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -42,7 +42,7 @@ tests/scripts/ Build, wheel, test, and documentation entry points docs/ Sphinx Python reference and autosummary templates cmake/ Build modules, dependency finders, and package configuration 3rdparty/ TVM FFI submodule and compiler support code -ci/jenkins/ Linux CPU, GPU, and ARM pipelines +ci/jenkins/ Unified Linux ARM, GPU, and EXTRA pipeline ci/wheel/ Wheel-build support .github/ GitHub Actions and reusable build actions ``` @@ -212,8 +212,9 @@ qualified and related namespaces adjacent in the Python API navigation. ## CI GitHub Actions runs macOS and Windows checks and a separate lint workflow. -Jenkins runs Linux CPU, GPU, and ARM builds and tests. The GPU pipeline builds -the documentation alongside tests after the wheel build. The executable entry +Jenkins uses `ci/jenkins/jenkinsfile.groovy` for parallel ARM, GPU, and EXTRA +branches. GPU builds one CUDA wheel for CPU tests, GPU tests, and documentation; +EXTRA independently builds the alternate GPU backends without tests. The executable entry points are in `tests/scripts/`; consult them for the exact CI environment. ## Further reading diff --git a/ci/jenkins/README.md b/ci/jenkins/README.md new file mode 100644 index 000000000..5031563f9 --- /dev/null +++ b/ci/jenkins/README.md @@ -0,0 +1,45 @@ +<!--- Licensed to the Apache Software Foundation (ASF) under one --> +<!--- or more contributor license agreements. See the NOTICE file --> +<!--- distributed with this work for additional information --> +<!--- regarding copyright ownership. The ASF licenses this file --> +<!--- to you under the Apache License, Version 2.0 (the --> +<!--- "License"); you may not use this file except in compliance --> +<!--- with the License. You may obtain a copy of the License at --> + +<!--- http://www.apache.org/licenses/LICENSE-2.0 --> + +<!--- Unless required by applicable law or agreed to in writing, --> +<!--- software distributed under the License is distributed on an --> +<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY --> +<!--- KIND, either express or implied. See the License for the --> +<!--- specific language governing permissions and limitations --> +<!--- under the License. --> + +# Jenkins + +Configure the TIRx multibranch job with script path +`ci/jenkins/jenkinsfile.groovy` and its Git SCM checkout. Retire the old +ARM/CPU/GPU jobs when switching to this job; their script paths have been +removed. Updating this repository does not change controller configuration. + +After preparation, ARM, GPU and EXTRA run in parallel: + +- ARM builds wheels and runs C++ and Rust tests in stock aarch64 manylinux. +- GPU builds CUDA wheels on a CPU node, then runs CPU tests, GPU tests and docs + in parallel using those exact FFI and TIRx wheels. All four stages use the + CUDA manylinux image's CPython 3.12; only GPU tests request a GPU device. + CPU tests exercise the CUDA wheel, replacing the separate CPU-only x86 wheel + build. C++ and Rust checks run during the wheel build; both Python suites + retain their examples. +- EXTRA independently builds source FFI and a wheel with OpenCL, Vulkan and ROCm + enabled, without tests. Its `ci_extra` mapping is the only legacy TVM CI image. + +The image defaults are in `docker-images.ini`; Jenkins parameters can override +individual images. Branch builds are needed to exercise edits to trusted CI +helpers. Verify the console's loaded script path and source commit when +checking a pipeline migration. + +Artifacts are isolated by job, build number and branch of the pipeline. GPU +consumers clear old wheels and verify the build's source tree and checksums. +Documentation deployment follows all successful joins and requires `main`, +no pull request, and `DOCS_DEPLOY_ENABLED=yes`; branch builds cannot deploy. diff --git a/ci/jenkins/arm_jenkinsfile.groovy b/ci/jenkins/arm_jenkinsfile.groovy deleted file mode 100644 index 000df1c78..000000000 --- a/ci/jenkins/arm_jenkinsfile.groovy +++ /dev/null @@ -1,46 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -// Reject untrusted helper edits before allocating nodes or running pipeline steps. -def helper_source -try { - helper_source = readTrusted('ci/jenkins/common.groovy') -} catch (Throwable ex) { - echo 'Jenkinsfile changes must be pushed to an upstream branch. ' + - 'arm/pr-head will fail for these changes; check arm/branch status instead.' - throw ex -} -ci = evaluate(helper_source) -ci.start() - -stage('Build') { - ci.with_node(instance: 'ARM-GRAVITON3-SPOT', retry_instance: 'ARM-GRAVITON3', - name: 'BUILD: arm', folder: 'tvm/build-arm', variables: ['PLATFORM=arm']) { - ci.docker_init(images.ci_arm) - withEnv(ci.build_environment()) { - sh( - script: """${docker_run} ${images.ci_arm} bash -euo pipefail -c ' - source ./tests/scripts/task_install_build_dependencies.sh - ./tests/scripts/task_build_arm.sh - '""", - label: 'Build arm wheels and C++ tests', - ) - } - ci.upload_artifacts('arm', '--path "dist/*.whl"') - - } -} diff --git a/ci/jenkins/common.groovy b/ci/jenkins/common.groovy index c386ab72c..b48f732c3 100644 --- a/ci/jenkins/common.groovy +++ b/ci/jenkins/common.groovy @@ -19,7 +19,7 @@ import org.jenkinsci.plugins.pipeline.modeldefinition.Utils // ------ Config ------ // Keep image names aligned with docker-images.ini. Tags are resolved in prepare. -image_platforms = [ci_arm: 'ARM', ci_cpu: 'CPU', ci_gpu: 'GPU'] +image_platforms = [ci_arm: 'ARM', ci_gpu: 'GPU', ci_extra: 'EXTRA'] // Resolve Hub inputs through the caller account ECR cache; false keeps public names. enable_docker_hub_cache = true max_time = 180 @@ -34,16 +34,16 @@ jenkins_scripts_root = 'ci/jenkins/scripts' images = image_platforms.collectEntries { name, platform -> [(name): ''] } source_images = [:] upstream_revision = null -s3_prefix = "tvm/${env.BRANCH_NAME}/${env.BUILD_NUMBER}" +s3_prefix = "tvm/${env.JOB_NAME}/${env.BUILD_NUMBER}" properties([parameters(image_platforms.collect { name, platform -> string(name: "${name}_param", defaultValue: '') })]) -// Shared utilities; platform Jenkinsfiles own stages, parallel groups and commands. +// Shared utilities; the Jenkinsfile owns stages, parallel groups and commands. def per_exec_ws(folder) { - return "workspace/exec_${env.EXECUTOR_NUMBER}/" + folder + return "workspace/${env.JOB_NAME}/exec_${env.EXECUTOR_NUMBER}/" + folder } def update_upstream_revision(git_ref) { @@ -219,7 +219,7 @@ def should_skip_ci(pr_number) { def prepare(node_type) { stage('Prepare') { node(node_type) { - ws("workspace/exec_${env.EXECUTOR_NUMBER}/tvm/prepare") { + ws(per_exec_ws('tvm/prepare')) { init_git() images.keySet().each { name -> @@ -330,23 +330,25 @@ def download_artifacts(tag) { ) } +// A failed/partial transfer must not reuse artifacts from an earlier build. +def download_gpu_wheels() { + sh 'rm -rf dist' + download_artifacts('gpu') + sh( + script: ''' + set -eu + test "$(cat dist/source-tree)" = "$(git rev-parse HEAD^{tree})" + sha256sum -c dist/SHA256SUMS + ''', + label: 'Verify CUDA wheel source and checksums', + ) +} + def docker_init(image) { // Forget compiler wrappers cached before the temporary sccache bypass. sh 'rm -f build/CMakeCache.txt 3rdparty/tvm-ffi/build/CMakeCache.txt' - // Clear out all Docker images that aren't going to be used - sh( - script: """ - set -eux - docker image ls --all - IMAGES=\$(docker image ls --all --format '{{.Repository}}:{{.Tag}} {{.ID}}') - - echo -e "Found images:\\n\$IMAGES" - echo "\$IMAGES" | { grep -vE '${image}' || test \$? = 1; } | { xargs docker rmi || test \$? = 123; } - - docker image ls --all - """, - label: 'Clean old Docker images', - ) + // Other parallel branches may have pulled an image but not started it yet. + // Do not remove images from the shared daemon during a build. if (ecr_registry(image)) { ecr_pull(image) diff --git a/ci/jenkins/cpu_jenkinsfile.groovy b/ci/jenkins/cpu_jenkinsfile.groovy deleted file mode 100644 index 54f149fa9..000000000 --- a/ci/jenkins/cpu_jenkinsfile.groovy +++ /dev/null @@ -1,66 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -// Reject untrusted helper edits before allocating nodes or running pipeline steps. -def helper_source -try { - helper_source = readTrusted('ci/jenkins/common.groovy') -} catch (Throwable ex) { - echo 'Jenkinsfile changes must be pushed to an upstream branch. ' + - 'cpu/pr-head will fail for these changes; check cpu/branch status instead.' - throw ex -} -ci = evaluate(helper_source) -ci.start() - -stage('Build') { - ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', - name: 'BUILD: CPU', folder: 'tvm/build-cpu', variables: ['PLATFORM=cpu']) { - ci.docker_init(images.ci_cpu) - withEnv(ci.build_environment()) { - sh( - script: """${docker_run} ${images.ci_cpu} bash -euo pipefail -c ' - source ./tests/scripts/task_install_build_dependencies.sh - ./tests/scripts/task_build_cpu.sh - '""", - label: 'Build cpu wheels and C++ tests', - ) - } - ci.upload_artifacts('cpu', '--path "dist/*.whl"') - - } -} - -stage('Test') { - parallel('unittest: CPU': { - ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', - name: 'unittest: CPU', folder: 'tvm/ut-python-cpu', - variables: ['PLATFORM=cpu', 'TEST_STEP_NAME=unittest: CPU']) { - ci.docker_init(images.ci_cpu) - sh 'rm -rf dist' - ci.download_artifacts('cpu') - sh ( - script: """${docker_run} ${images.ci_cpu} bash -euo pipefail -c ' - source ./tests/scripts/test_task_install_dist_wheels.sh - ./tests/scripts/task_python_unittest.sh - ./tests/scripts/task_minimal_examples_cpu.sh - '""", - label: 'Run Python unit tests', - ) - } - }) -} diff --git a/ci/jenkins/docker-images.ini b/ci/jenkins/docker-images.ini index fbd96a177..9ce9a3731 100644 --- a/ci/jenkins/docker-images.ini +++ b/ci/jenkins/docker-images.ini @@ -18,6 +18,8 @@ # This data file is read during when Jenkins runs job to determine docker images. [jenkins] ci_tag: 20260705-142349-cfb98e93 -ci_arm: tlcpack/ci-arm:%(ci_tag)s -ci_cpu: tlcpack/ci-cpu:%(ci_tag)s -ci_gpu: tlcpack/ci-gpu:%(ci_tag)s +ci_arm: quay.io/pypa/manylinux_2_28_aarch64:latest +ci_cpu: quay.io/pypa/manylinux_2_28_x86_64:latest +ci_gpu: quay.io/manylinux_cuda/manylinux_2_28_x86_64_cuda13_1:latest +# Only EXTRA needs the legacy OpenCL/Vulkan/ROCm toolchains. +ci_extra: tlcpack/ci-gpu:%(ci_tag)s diff --git a/ci/jenkins/docker/bash.sh b/ci/jenkins/docker/bash.sh index 0e7064d07..8bc412f63 100755 --- a/ci/jenkins/docker/bash.sh +++ b/ci/jenkins/docker/bash.sh @@ -591,19 +591,27 @@ else rm /this_is_writable_file_system fi -getent group "${CI_BUILD_GID}" || ( - # Ensure "${CI_BUILD_GROUP}" is not already some other gid inside container. - if grep -q "^${CI_BUILD_GROUP}:" /etc/group; then - CI_BUILD_GROUP="${CI_BUILD_GROUP}2" - fi - addgroup --force-badname --gid "${CI_BUILD_GID}" "${CI_BUILD_GROUP}" >/dev/null) - -getent group tvm-venv || (addgroup tvm-venv >/dev/null) -getent passwd "${CI_BUILD_UID}" || adduser --force-badname --gid "${CI_BUILD_GID}" --uid "${CI_BUILD_UID}" \ - --gecos "${CI_BUILD_USER} (generated by with_the_same_user script)" \ - --disabled-password --home "${CI_BUILD_HOME}" --quiet "${CI_BUILD_USER}" -usermod -a -G sudo -G tvm-venv "${CI_BUILD_USER}" -usermod -a -G sudo -G dialout "${CI_BUILD_USER}" +if command -v addgroup >/dev/null; then + getent group "${CI_BUILD_GID}" || ( + # Ensure "${CI_BUILD_GROUP}" is not already some other gid inside container. + if grep -q "^${CI_BUILD_GROUP}:" /etc/group; then + CI_BUILD_GROUP="${CI_BUILD_GROUP}2" + fi + addgroup --force-badname --gid "${CI_BUILD_GID}" "${CI_BUILD_GROUP}" >/dev/null) + + getent group tvm-venv || (addgroup tvm-venv >/dev/null) + getent passwd "${CI_BUILD_UID}" || adduser --force-badname --gid "${CI_BUILD_GID}" --uid "${CI_BUILD_UID}" \ + --gecos "${CI_BUILD_USER} (generated by with_the_same_user script)" \ + --disabled-password --home "${CI_BUILD_HOME}" --quiet "${CI_BUILD_USER}" + usermod -a -G sudo -G tvm-venv "${CI_BUILD_USER}" + usermod -a -G sudo -G dialout "${CI_BUILD_USER}" +else + # Stock manylinux images use the shadow-utils commands, not Debian adduser. + dnf -y install sudo + getent group "${CI_BUILD_GID}" || groupadd --gid "${CI_BUILD_GID}" "${CI_BUILD_GROUP}" + getent passwd "${CI_BUILD_UID}" || useradd --gid "${CI_BUILD_GID}" --uid "${CI_BUILD_UID}" \ + --home-dir "${CI_BUILD_HOME}" --create-home "${CI_BUILD_USER}" +fi # Add user to video group for ROCm if [[ ! -z "${ROCM_ENABLED-}" ]]; then diff --git a/ci/jenkins/gpu_jenkinsfile.groovy b/ci/jenkins/gpu_jenkinsfile.groovy deleted file mode 100644 index e6d1c5789..000000000 --- a/ci/jenkins/gpu_jenkinsfile.groovy +++ /dev/null @@ -1,147 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -// Reject untrusted helper edits before allocating nodes or running pipeline steps. -def helper_source -try { - helper_source = readTrusted('ci/jenkins/common.groovy') -} catch (Throwable ex) { - echo 'Jenkinsfile changes must be pushed to an upstream branch. ' + - 'gpu/pr-head will fail for these changes; check gpu/branch status instead.' - throw ex -} -ci = evaluate(helper_source) -ci.start() - -stage('Build') { - ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', - name: 'BUILD: GPU', folder: 'tvm/build-gpu', variables: ['PLATFORM=gpu']) { - ci.docker_init(images.ci_gpu) - withEnv(ci.build_environment()) { - sh( - script: """${docker_run} --no-gpu ${images.ci_gpu} bash -euo pipefail -c ' - source ./tests/scripts/task_install_build_dependencies.sh - ./tests/scripts/task_build_gpu.sh - '""", - label: 'Build gpu', - ) - } - ci.upload_artifacts('gpu', '--path "dist/*.whl"') - - // Check the compiler-only GPU configuration after publishing the test build. - sh 'rm -rf build' - withEnv(ci.build_environment()) { - sh( - script: "${docker_run} --no-gpu ${images.ci_gpu} ./tests/scripts/task_build_gpu_other.sh", - label: 'Build gpu_other', - ) - } - } -} - -parallel( - 'Test': { - stage('Test') { - ci.with_node(instance: 'GPU-SPOT', retry_instance: 'GPU', - name: 'unittest: GPU', folder: 'tvm/ut-python-gpu', - variables: ['PLATFORM=gpu', 'TEST_STEP_NAME=unittest: GPU']) { - ci.docker_init(images.ci_gpu) - sh 'rm -rf dist' - ci.download_artifacts('gpu') - sh( - script: """${docker_run} ${images.ci_gpu} bash -euo pipefail -c ' - source ./tests/scripts/test_task_install_dist_wheels.sh --group cuda - ./tests/scripts/task_python_unittest_gpuonly.sh - ./tests/scripts/task_minimal_examples_cuda.sh - '""", - label: 'Run Python GPU unit tests', - ) - } - } - }, - 'Docs': { - stage('Docs') { - ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', - name: 'Docs', folder: 'tvm/tirx-docs', variables: ['PLATFORM=gpu']) { - ci.docker_init(images.ci_gpu) - sh 'rm -rf dist' - ci.download_artifacts('gpu') - sh( - script: "${docker_run} --no-gpu ${images.ci_gpu} ./tests/scripts/task_python_docs.sh", - label: 'Build TIRx API documentation', - ) - ci.upload_artifacts('docs', '--path docs.tgz') - } - } - }, -) - -stage('Deploy Docs') { - if (env.BRANCH_NAME == 'main' && !env.CHANGE_ID && env.DOCS_DEPLOY_ENABLED == 'yes') { - ci.with_node(instance: 'CPU', name: 'Deploy Docs', folder: 'tvm/deploy-tirx-docs') { - ci.download_artifacts('docs') - // Keep credential-bearing deployment in the trusted Jenkinsfile. - sh( - script: ''' - set -eu - SOURCE_COMMIT=$(git rev-parse HEAD) - rm -rf tvm-site - git clone --branch asf-site --depth=1 https://github.com/apache/tvm-site.git tvm-site - cd tvm-site - git log -1 --format='Site base: %H' - rm -rf tirx - mkdir tirx - tar -xzf ../docs.tgz -C tirx - test -s tirx/index.html - test "$(cat tirx/commit_hash)" = "$SOURCE_COMMIT" - git config user.name tvm-bot - git config user.email [email protected] - git add -f -- tirx - test -z "$(git diff --cached --name-only -- . ':!:tirx')" - git diff --cached --shortstat - git diff --cached --name-only | sed -n '1,20p' - if ! git diff --cached --quiet; then - git commit --quiet -m "deploying tirx docs ($(cat tirx/commit_hash))" -- tirx - fi - git log -1 --format='Docs preview commit: %H' - ''', - label: 'Update tvm-site/tirx', - ) - withCredentials([string(credentialsId: 'docs-push-token', variable: 'GITHUB_TOKEN')]) { - sh( - script: ''' - set +x - set -eu - cd tvm-site - auth=$(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\\n') - for attempt in 1 2 3; do - if git -c http.extraHeader="Authorization: Basic $auth" push origin HEAD:asf-site; then - exit 0 - fi - if [ "$attempt" = 3 ]; then - exit 1 - fi - sleep 2 - git pull --rebase origin asf-site - done - ''', - label: 'Publish TIRx documentation to apache/tvm-site', - ) - } - } - } -} diff --git a/ci/jenkins/jenkinsfile.groovy b/ci/jenkins/jenkinsfile.groovy new file mode 100644 index 000000000..17cf97c96 --- /dev/null +++ b/ci/jenkins/jenkinsfile.groovy @@ -0,0 +1,199 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +// Reject untrusted helper edits before allocating nodes or running pipeline steps. +def helper_source +try { + helper_source = readTrusted('ci/jenkins/common.groovy') +} catch (Throwable ex) { + echo 'Jenkinsfile changes must be pushed to an upstream branch. ' + + 'Check the branch build when PR helpers differ from the trusted target.' + throw ex +} +ci = evaluate(helper_source) +ci.start() + +parallel( + 'ARM': { + stage('ARM') { + ci.with_node(instance: 'ARM-GRAVITON3-SPOT', retry_instance: 'ARM-GRAVITON3', + name: 'ARM', folder: 'tvm/build-arm', variables: ['PLATFORM=arm']) { + ci.docker_init(images.ci_arm) + withEnv(ci.build_environment()) { + sh( + script: """${docker_run} --no-gpu ${images.ci_arm} bash -euo pipefail -c ' + source ./tests/scripts/task_install_build_dependencies.sh + ./tests/scripts/task_build_arm.sh + '""", + label: 'Build ARM wheels and run C++/Rust tests', + ) + } + ci.upload_artifacts('arm', '--path "dist/*.whl"') + } + } + }, + 'GPU': { + stage('BUILD: GPU') { + ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', + name: 'BUILD: GPU', folder: 'tvm/build-gpu', variables: ['PLATFORM=gpu']) { + ci.docker_init(images.ci_gpu) + withEnv(ci.build_environment()) { + sh( + script: """${docker_run} --no-gpu ${images.ci_gpu} bash -euo pipefail -c ' + source ./tests/scripts/task_install_build_dependencies.sh + ./tests/scripts/task_build_gpu.sh + '""", + label: 'Build CUDA wheels and run C++/Rust tests', + ) + } + // PR merge commit IDs can differ between workspaces; compare source trees. + sh 'git rev-parse HEAD^{tree} > dist/source-tree; sha256sum dist/*.whl > dist/SHA256SUMS' + ci.upload_artifacts('gpu', '--path "dist/*"') + } + } + // All consumers use this build's exact FFI and CUDA wheels under CPython 3.12. + parallel( + 'UT: CPU': { + stage('UT: CPU') { + ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', + name: 'UT: CPU', folder: 'tvm/ut-python-cpu', + variables: ['PLATFORM=cpu', 'TEST_STEP_NAME=UT: CPU']) { + ci.docker_init(images.ci_gpu) + ci.download_gpu_wheels() + sh( + script: """${docker_run} --no-gpu ${images.ci_gpu} bash -euo pipefail -c ' + source ./tests/scripts/test_task_install_dist_wheels.sh + ./tests/scripts/task_python_unittest.sh + ./tests/scripts/task_minimal_examples_cpu.sh + '""", + label: 'Run CPU unit tests and examples on the CUDA wheel', + ) + } + } + }, + 'UT: GPU': { + stage('UT: GPU') { + ci.with_node(instance: 'GPU-SPOT', retry_instance: 'GPU', + name: 'UT: GPU', folder: 'tvm/ut-python-gpu', + variables: ['PLATFORM=gpu', 'TEST_STEP_NAME=UT: GPU']) { + ci.docker_init(images.ci_gpu) + ci.download_gpu_wheels() + sh( + script: """${docker_run} ${images.ci_gpu} bash -euo pipefail -c ' + export LIBRARY_PATH=/usr/local/cuda/lib64/stubs + source ./tests/scripts/test_task_install_dist_wheels.sh --group cuda + ./tests/scripts/task_python_unittest_gpuonly.sh + ./tests/scripts/task_minimal_examples_cuda.sh + '""", + label: 'Run GPU unit tests and examples', + ) + } + } + }, + 'DOCS': { + stage('DOCS') { + ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', + name: 'DOCS', folder: 'tvm/tirx-docs', variables: ['PLATFORM=gpu']) { + ci.docker_init(images.ci_gpu) + ci.download_gpu_wheels() + sh( + script: "${docker_run} --no-gpu ${images.ci_gpu} ./tests/scripts/task_python_docs.sh", + label: 'Build TIRx API documentation', + ) + ci.upload_artifacts('docs', '--path docs.tgz') + } + } + }, + ) + }, + 'EXTRA': { + stage('EXTRA') { + ci.with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', + name: 'EXTRA', folder: 'tvm/build-extra', variables: ['PLATFORM=extra']) { + ci.docker_init(images.ci_extra) + withEnv(ci.build_environment()) { + sh( + script: """${docker_run} --no-gpu ${images.ci_extra} bash -euo pipefail -c ' + source ./tests/scripts/task_install_build_dependencies.sh + ./tests/scripts/task_build_gpu_other.sh + '""", + label: 'Build OpenCL, Vulkan and ROCm wheels without tests', + ) + } + ci.upload_artifacts('extra', '--path "dist/*.whl"') + } + } + }, +) + +// Deployment requires all three branches and every GPU consumer to succeed. +stage('Deploy Docs') { + if (env.BRANCH_NAME == 'main' && !env.CHANGE_ID && env.DOCS_DEPLOY_ENABLED == 'yes') { + ci.with_node(instance: 'CPU', name: 'Deploy Docs', folder: 'tvm/deploy-tirx-docs') { + sh 'rm -f docs.tgz' + ci.download_artifacts('docs') + // Keep credential-bearing deployment in the trusted Jenkinsfile. + sh( + script: ''' + set -eu + SOURCE_COMMIT=$(git rev-parse HEAD) + rm -rf tvm-site + git clone --branch asf-site --depth=1 https://github.com/apache/tvm-site.git tvm-site + cd tvm-site + git log -1 --format='Site base: %H' + rm -rf tirx + mkdir tirx + tar -xzf ../docs.tgz -C tirx + test -s tirx/index.html + test "$(cat tirx/commit_hash)" = "$SOURCE_COMMIT" + git config user.name tvm-bot + git config user.email [email protected] + git add -f -- tirx + test -z "$(git diff --cached --name-only -- . ':!:tirx')" + git diff --cached --shortstat + git diff --cached --name-only | sed -n '1,20p' + if ! git diff --cached --quiet; then + git commit --quiet -m "deploying tirx docs ($(cat tirx/commit_hash))" -- tirx + fi + git log -1 --format='Docs preview commit: %H' + ''', + label: 'Update tvm-site/tirx', + ) + withCredentials([string(credentialsId: 'docs-push-token', variable: 'GITHUB_TOKEN')]) { + sh( + script: ''' + set +x + set -eu + cd tvm-site + auth=$(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\\n') + for attempt in 1 2 3; do + if git -c http.extraHeader="Authorization: Basic $auth" push origin HEAD:asf-site; then + exit 0 + fi + if [ "$attempt" = 3 ]; then + exit 1 + fi + sleep 2 + git pull --rebase origin asf-site + done + ''', + label: 'Publish TIRx documentation to apache/tvm-site', + ) + } + } + } +} diff --git a/ci/jenkins/scripts/determine_docker_images.py b/ci/jenkins/scripts/determine_docker_images.py index 893ac397e..4880911d4 100755 --- a/ci/jenkins/scripts/determine_docker_images.py +++ b/ci/jenkins/scripts/determine_docker_images.py @@ -137,8 +137,8 @@ if __name__ == "__main__": name_dir.mkdir(exist_ok=True) images_to_use = {} for filename, spec in images.items(): - if spec.startswith(TVM_CI_ECR): - logging.info(f"{spec} is from ECR") + if spec.startswith((TVM_CI_ECR, "quay.io/")): + logging.info(f"{spec} uses its explicit registry") images_to_use[filename] = spec elif image_exists(spec): logging.info(f"{spec} found in tlcpack") diff --git a/tests/scripts/task_build_gpu.sh b/tests/scripts/task_build_gpu.sh index 45f20bd58..ad6044420 100755 --- a/tests/scripts/task_build_gpu.sh +++ b/tests/scripts/task_build_gpu.sh @@ -18,8 +18,11 @@ set -euxo pipefail +source ./tests/scripts/task_install_rust.sh + export CMAKE_ARGS=" -DTVM_TIRX_BUILD_TESTS=ON + -DTVM_TIRX_USE_RUST=ON -DTVM_TIRX_USE_CUDA=ON -DTVM_TIRX_USE_VULKAN=OFF -DTVM_TIRX_USE_OPENCL=OFF @@ -29,3 +32,7 @@ export CMAKE_ARGS=" uv build --wheel --no-build-isolation --offline --out-dir dist . ./build/tirx_cpptest + +# Retain the x86 Rust checks with the wheel used by both CPU and GPU tests. +PATH="$PWD/build/rust/bin:$PATH" cargo test --manifest-path rust/Cargo.toml +uv run --no-sync python -m tvm_ffi.stub.cli rust/src --target rust --dlls build/lib/libtvm_tirx_compiler.so --check diff --git a/tests/scripts/task_build_gpu_other.sh b/tests/scripts/task_build_gpu_other.sh index caa9dcd58..26a75fb03 100755 --- a/tests/scripts/task_build_gpu_other.sh +++ b/tests/scripts/task_build_gpu_other.sh @@ -16,16 +16,16 @@ # specific language governing permissions and limitations # under the License. -# Compile the backends excluded from the GPU test wheel without running them. +# Build the alternate backend wheel independently of the CUDA test artifacts. set -euxo pipefail -# Keep these libraries separate from the published GPU wheel artifacts. -unset PYTHONPATH TVM_LIBRARY_PATH TVM_FFI_LIBRARY_PATH TVM_TIRX_LIBRARY_PATH -export PYTHONNOUSERSITE=1 -source .venv/bin/activate -uv sync --only-group build -uv pip install --no-deps --offline --reinstall dist/apache_tvm_ffi-*.whl -uv run --no-sync cmake -S . -B build -GNinja -DCMAKE_BUILD_TYPE=RelWithDebInfo \ - -DTVM_TIRX_USE_OPENCL=ON -DTVM_TIRX_USE_VULKAN=ON -DTVM_TIRX_USE_ROCM=ON \ +export CMAKE_ARGS=" + -DTVM_TIRX_BUILD_TESTS=OFF + -DTVM_TIRX_USE_RUST=OFF + -DTVM_TIRX_USE_CUDA=OFF + -DTVM_TIRX_USE_OPENCL=ON + -DTVM_TIRX_USE_VULKAN=ON + -DTVM_TIRX_USE_ROCM=ON -DCMAKE_CXX_FLAGS=-Werror -uv run --no-sync cmake --build build --target tvm_tirx_compiler tvm_runtime_opencl tvm_runtime_vulkan tvm_runtime_rocm +" +uv build --wheel --no-build-isolation --offline --out-dir dist . diff --git a/tests/scripts/task_install_build_dependencies.sh b/tests/scripts/task_install_build_dependencies.sh index 0beed707f..524bb17f3 100755 --- a/tests/scripts/task_install_build_dependencies.sh +++ b/tests/scripts/task_install_build_dependencies.sh @@ -18,6 +18,14 @@ set -euxo pipefail +# Use the same bundled interpreter and toolchain for manylinux builds and consumers. +if [ -d /opt/python ]; then + export PATH="/opt/python/cp312-cp312/bin:/opt/rh/gcc-toolset-14/root/usr/bin:/usr/local/cuda/bin:$HOME/.local/bin:$PATH" + test -x /opt/python/cp312-cp312/bin/python3 + export CC=gcc CXX=g++ + python3 -m pip install --user uv +fi + rm -rf dist unset PYTHONPATH TVM_LIBRARY_PATH TVM_FFI_LIBRARY_PATH TVM_TIRX_LIBRARY_PATH export PYTHONNOUSERSITE=1 diff --git a/tests/scripts/task_python_docs.sh b/tests/scripts/task_python_docs.sh index d07518080..0217b244a 100755 --- a/tests/scripts/task_python_docs.sh +++ b/tests/scripts/task_python_docs.sh @@ -19,6 +19,8 @@ set -euxo pipefail source ./tests/scripts/test_task_install_dist_wheels.sh --group docs +# Keep function-pointer XML compatible with Breathe. +uv pip install mlc-doxygen-bin==1.17.0 # cargo for the Rust API reference; the crate's build script finds the installed wheel's # libtvm_tirx_compiler through the tvm-tirx-config of the activated .venv. # Rust 1.84's rustdoc crashes on proc-macro doc links (rust-lang/rust#134715). diff --git a/tests/scripts/test_task_install_dist_wheels.sh b/tests/scripts/test_task_install_dist_wheels.sh index 0ccbe1c92..b3b90fd25 100755 --- a/tests/scripts/test_task_install_dist_wheels.sh +++ b/tests/scripts/test_task_install_dist_wheels.sh @@ -18,6 +18,14 @@ set -euxo pipefail +# Use the same bundled interpreter and toolchain for manylinux builds and consumers. +if [ -d /opt/python ]; then + export PATH="/opt/python/cp312-cp312/bin:/opt/rh/gcc-toolset-14/root/usr/bin:/usr/local/cuda/bin:$HOME/.local/bin:$PATH" + test -x /opt/python/cp312-cp312/bin/python3 + export CC=gcc CXX=g++ + python3 -m pip install --user uv +fi + unset PYTHONPATH TVM_LIBRARY_PATH TVM_FFI_LIBRARY_PATH TVM_TIRX_LIBRARY_PATH export PYTHONNOUSERSITE=1 uv venv --clear --python "$(command -v python3)"
