The GitHub Actions job "Lint" on tvm.git/main has failed. Run started by GitHub user mshr-h (triggered by mshr-h).
Head commit for run: b473185418cc8e7c27590d3294bc86d97cc8027d / HoYi <[email protected]> [Relax][ONNX] Add ONNX Backend Tests for systematic frontend coverage (#19515) ## Summary Introduce a test runner that reuses the official ONNX Backend Test Suite to systematically verify the Relax ONNX importer. This complements the existing hand-written tests in `test_frontend_onnx.py` by providing spec-aligned coverage of standard ONNX operator semantics. Towards #19505 ## Motivation The existing `test_frontend_onnx.py` has 187 hand-written tests that validate TVM-specific importer behavior (parameter handling, name sanitization, dynamic shapes, Relax IR structure). However, it relies on ONNX Runtime as the reference and cannot systematically cover all edge cases defined in the ONNX specification. The ONNX Backend Test Suite provides 1653+ node-level tests with protobuf reference inputs/outputs. It is the industry standard for validating ONNX importers/exporters (used by ONNX Runtime, TensorFlow, PyTorch). Reusing it gives Relax a living, upstream-aligned correctness baseline. ## What this PR adds - `tests/python/relax/test_frontend_onnx_backend.py` — a backend adapter (`TVMRelaxBackend`) that implements the `onnx.backend.base.Backend` interface, wiring `from_onnx()` → `DecomposeOpsForInference()` → `LegalizeOps()` → `tvm.compile()` → `VirtualMachine`. ## Coverage 72 operators with 388 test cases, all passing. Only operators where every ONNX node test passes are included — no xfail markers. Operators not yet covered include: cast (exotic dtypes), reduce ops (edge cases), reshape/resize/attention (complex behavior), quantization, and several others with known importer gaps. These can be added incrementally as the importer improves. ## Test results 388 passed, 3216 skipped (CUDA variants + operators not yet in allowlist), 0 failed, 0 xfailed ## CI impact - New test file is not added to any existing CI test shard by default - Full suite (388 tests) is lightweight on CPU-only runners ## Design decisions - **Coexistence with existing tests**: `test_frontend_onnx.py` remains unchanged. Backend tests cover standard ONNX semantics; hand-written tests continue to cover TVM-specific behavior (dynamic shapes, Relax IR structure, importer options). - **Public API only**: uses `backend_test.include()` with `^`-anchored regex patterns. No access to private ONNX APIs. - **No xfail**: only include operators that fully pass. Uncovered operators are documented in code comments and this PR description. Follow-up PRs can expand coverage as importer gaps are fixed. - **Prefix conflict handling**: `include()` patterns use `^test_{op}(?:_.*)?(?:_cpu|_cuda)$`, which can cause false matches when a short op name is a prefix of a longer one (e.g. `log` vs `log_softmax`). Affected ops (`log`, `max`, `relu`) are excluded until a more precise matching strategy is adopted. Report URL: https://github.com/apache/tvm/actions/runs/25801150566 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
