mshr-h opened a new issue, #19505: URL: https://github.com/apache/tvm/issues/19505
This is my brain dump for a possible improvement to the Relax ONNX frontend tests. Currently, many ONNX frontend tests in Relax manually construct small ONNX models and compare the imported Relax program against ONNX Runtime. This makes the test suite somewhat ad-hoc. As coverage grows, it becomes easy to miss ONNX spec corner cases, opset-specific behavior, and edge cases that are already covered by the official ONNX test suite. ONNX provides an official Backend Test suite, including node-level tests for operator behavior: - https://github.com/onnx/onnx/blob/main/docs/OnnxBackendTest.md I think it would be worth exploring whether Relax's ONNX frontend tests can reuse those tests as a more systematic source of coverage. The goal is not necessarily to expose TVM as a full ONNX backend. Rather, the idea is to use the ONNX Backend Tests as a testing mechanism for the importer: - improve coverage of ONNX operator semantics - reduce duplicated hand-written operator tests - catch edge cases that are easy to miss with random-input tests - align frontend correctness testing more closely with the ONNX specification - keep TVM-specific frontend tests only for behavior that ONNX Backend Tests do not cover, such as importer options, parameter handling, name sanitization, and Relax-specific integration I do not currently have enough time to work on this myself, so contributions, suggestions, or alternative ideas would be very welcome. This issue is intended to collect thoughts and discuss whether this direction makes sense before working out a detailed design. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
