| Issue |
129443
|
| Summary |
[mlir] Testing infra improvements ideas
|
| Labels |
mlir
|
| Assignees |
banach-space
|
| Reporter |
banach-space
|
Meta-ticket to collect ideas for improving the testing infrastructure in MLIR.
At the moment, this is a collection of suggestions extracted from:
* https://discourse.llvm.org/t/rfc-should-we-aim-for-more-consistency-in-tests/, and
* https://github.com/llvm/mlir-www/pull/216.
The order of these items is random. I will try to add more data/notes over time.
_Please, if you decide to work on any of these, leave a comment so that we avoid duplicating efforts._
### IDEA 1
Separate examples from ODS definitions into a new field: `let examples = {}`. Build infra so that `ninja check-mlir` would extra those examples and sanity-check them with `mlir-opt` (to avoid the examples becoming out-dated).
This could be used to demonstrate both "valid" and invalid usage.
### IDEA 2
Generate, somehow, examples from the ODS `assemblyFormat` (even if we will likely want to update them manually).
### IDEA 3
Revisit the concept of round-trip tests in "ops.mlir" files. They shouldn't be strictly necessary for ops with `let assemblyFormat` MLIR thoroughly tests the printer/parser generator.
### IDEA 4
Improve "split" locations when encountering errors. Currently these are OK for humans to read - make them easier for tools to parse (and to jump to locations).
### IDEA 5
Add more metadata to “splits” so that one could do something like:
```
---- Test: Verify the folding behavior with single dynamic rank.
```
This would then be printed alongside the error/failure message. This should be integrated with the existing metadata like [here](https://github.com/llvm/llvm-project/blob/1aea0241f1cce9eb4eba3e4add3be9370e30e415/mlir/test/Dialect/Vector/vector-transfer-permutation-lowering.mlir#L8-L13).
### IDEA 6
Make [generate-test-checks.py](https://github.com/llvm/llvm-project/blob/main/mlir/utils/generate-test-checks.py) generate better names. For example:
```mlir
// Automatically generated - possible near-future version (mild improvement)
// CHECK-LABEL: func.func @bitcast_2d(
// CHECK-SAME: %[[ARG_0]]: vector<2x4xi32>) -> vector<2x16xi8> {
// CHECK: %[[BITCAST_1:.*]] = vector.bitcast %[[ARG_0]] : vector<2x4xi32> to vector<2x2xi64>
// CHECK: %[[BITCAST_2.:*]] = vector.bitcast %[[BITCAST_1]] : vector<2x2xi64> to vector<2x16xi8>
// CHECK: return %[[BITCAST_2]] : vector<2x16xi8>
func.func @bitcast_2d(%arg0: vector<2x4xi32>) -> vector<2x16xi8> {
%0 = vector.bitcast %arg0 : vector<2x4xi32> to vector<2x2xi64>
%1 = vector.bitcast %0 : vector<2x2xi64> to vector<2x16xi8>
return %1 : vector<2x16xi8>
}
```
instead of:
```mlir
// Automatically generated - today’s version
// CHECK-LABEL: func.func @bitcast_2d(
// CHECK-SAME: %[[VAL_0:[0-9]+|[a-zA-Z$._-][a-zA-Z0-9$._-]*]]: vector<2x4xi32>) -> vector<2x16xi8> {
// CHECK: %[[VAL_1:.*]] = vector.bitcast %[[VAL_0]] : vector<2x4xi32> to vector<2x2xi64>
// CHECK: %[[VAL_2:.*]] = vector.bitcast %[[VAL_1]] : vector<2x2xi64> to vector<2x16xi8>
// CHECK: return %[[VAL_2]] : vector<2x16xi8>
func.func @bitcast_2d(%arg0: vector<2x4xi32>) -> vector<2x16xi8> {
%0 = vector.bitcast %arg0 : vector<2x4xi32> to vector<2x2xi64>
%1 = vector.bitcast %0 : vector<2x2xi64> to vector<2x16xi8>
return %1 : vector<2x16xi8>
}
```
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
