llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-mlir Author: Markus Böck (zero9178) <details> <summary>Changes</summary> Prior to this PR, ODS had one mechanism for sharding which equally split a `<Dialect>Ops.cpp.inc` file into `n` many TUs, as requested on the command line (using preprocessor defines for selective enablement). The motivation for doing so is that this can speed up compile time by splitting class definition compilation to multiple translation units. However, this approach has one large disadvantage: Since only the `<Dialect>Ops.cpp.inc` is split, the correpsonding `<Dialect>Ops.h.inc` is still included in each translation unit. For a large dialect like NVVM or LLVM this leads to **much higher** concurrent memory usage while compiling as every compilation process contains an in-memory copy of the header plus can lead to slower compilation time overall if too many shards are being used as overall work goes up (`n` header parses instead of 1). This PR tries to address these issues by introducing a new kind of sharding called "natural sharding". The idea is that we split `<Dialect>Ops.cpp.inc` and `<Dialect>Ops.h.inc` along so called partitions. A new header can be created for every partition and every translation unit only contains the code from its partition. This means a TU no longer needs to include the entire dialect header but only the header from its partition. One can think of these as effectively being sub-dialects from the perspective of the required boilerplate. We call it "Natural Sharding", as the partitions are defined by the dialect authors and are meant to be categories that naturally categorizes a dialects operations (e.g., Memory ops, control flow ops, arithmetic ops, intrinsic ops, etc. in the LLVM Dialect). The new `-op-partition` then filters output generation to just a given partition. This PR introduces the mechanism and for testing, already exercises it on the test dialect. Users of `TestOps.h` were further changed to just include a given partition if deemed useful. Overall this yielded the following build gains vs the existing sharding, measured on a 16 thread machine (Release + assertions, 16 parallel jobs, median of 3 runs). "Users of `MLIRTestDialect`" are other TUs (unittests mainly) that include `MLIRTestDialect` headers. Total memory is the sum of the max RSS of every TU: | TUs | Compiler | Wall time (s) | CPU time (s) | Total memory used during compilation (GiB) | |----------------------------|----------|---------------------|--------------------|--------------------------------------------| | `MLIRTestDialect` | clang | 49.9 -> 35.2 (-29%) | 710 -> 365 (-49%) | 23.3 -> 11.1 (-52%) | | `MLIRTestDialect` | clang -g | 57.8 -> 43.1 (-25%) | 791 -> 427 (-46%) | 25.0 -> 12.5 (-50%) | | `MLIRTestDialect` | gcc | 58.6 -> 43.8 (-25%) | 740 -> 395 (-47%) | 41.9 -> 19.4 (-54%) | | `MLIRTestDialect` | gcc -g | 82.5 -> 61.5 (-25%) | 1059 -> 570 (-46%) | 52.9 -> 26.7 (-50%) | | Users of `MLIRTestDialect` | clang | 45.4 -> 35.1 (-23%) | 626 -> 472 (-25%) | 27.6 -> 21.5 (-22%) | | Users of `MLIRTestDialect` | clang -g | 46.6 -> 36.1 (-23%) | 644 -> 493 (-23%) | 28.4 -> 22.3 (-21%) | | Users of `MLIRTestDialect` | gcc | 43.6 -> 33.9 (-22%) | 604 -> 449 (-26%) | 44.5 -> 32.8 (-26%) | | Users of `MLIRTestDialect` | gcc -g | 60.9 -> 46.1 (-24%) | 859 -> 616 (-28%) | 57.1 -> 42.4 (-26%) | Future plan is to introduce this for types and attributes as well --- Patch is 145.47 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/239254.diff 66 Files Affected: - (modified) mlir/docs/DefiningDialects/Operations.md (+104) - (modified) mlir/include/mlir/IR/DialectBase.td (+33) - (modified) mlir/include/mlir/IR/OpBase.td (+24) - (modified) mlir/include/mlir/TableGen/Dialect.h (+5) - (modified) mlir/include/mlir/TableGen/Operator.h (+3) - (modified) mlir/lib/TableGen/Dialect.cpp (+5) - (modified) mlir/lib/TableGen/Operator.cpp (+5) - (modified) mlir/test/lib/Analysis/DataFlow/TestDenseBackwardDataFlowAnalysis.cpp (+1-1) - (modified) mlir/test/lib/Analysis/DataFlow/TestDenseForwardDataFlowAnalysis.cpp (+1-2) - (modified) mlir/test/lib/Conversion/FuncToLLVM/TestConvertCallOp.cpp (+1-2) - (modified) mlir/test/lib/Dialect/Affine/TestReifyValueBounds.cpp (+1-2) - (modified) mlir/test/lib/Dialect/Bufferization/TestOneShotModuleBufferize.cpp (+1-2) - (modified) mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp (+1-1) - (modified) mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp (+1-2) - (modified) mlir/test/lib/Dialect/Test/CMakeLists.txt (+14-7) - (modified) mlir/test/lib/Dialect/Test/TestDialect.cpp (+1-2) - (modified) mlir/test/lib/Dialect/Test/TestDialect.td (+19-1) - (modified) mlir/test/lib/Dialect/Test/TestFromLLVMIRTranslation.cpp (+1-2) - (modified) mlir/test/lib/Dialect/Test/TestOps.h (+12-238) - (modified) mlir/test/lib/Dialect/Test/TestOps.td (+224-174) - (renamed) mlir/test/lib/Dialect/Test/TestOpsAttributes.cpp (+4-5) - (added) mlir/test/lib/Dialect/Test/TestOpsAttributes.h (+17) - (added) mlir/test/lib/Dialect/Test/TestOpsConversion.cpp (+15) - (added) mlir/test/lib/Dialect/Test/TestOpsConversion.h (+17) - (added) mlir/test/lib/Dialect/Test/TestOpsCore.cpp (+19) - (added) mlir/test/lib/Dialect/Test/TestOpsCore.h (+18) - (added) mlir/test/lib/Dialect/Test/TestOpsInterfaces.cpp (+16) - (added) mlir/test/lib/Dialect/Test/TestOpsInterfaces.h (+90) - (added) mlir/test/lib/Dialect/Test/TestOpsPatterns.cpp (+15) - (added) mlir/test/lib/Dialect/Test/TestOpsPatterns.h (+17) - (added) mlir/test/lib/Dialect/Test/TestOpsProperties.cpp (+16) - (added) mlir/test/lib/Dialect/Test/TestOpsProperties.h (+154) - (modified) mlir/test/lib/Dialect/Test/TestOpsSyntax.cpp (+2-8) - (modified) mlir/test/lib/Dialect/Test/TestOpsSyntax.td (+5-1) - (added) mlir/test/lib/Dialect/Test/TestOpsTraits.cpp (+15) - (added) mlir/test/lib/Dialect/Test/TestOpsTraits.h (+17) - (modified) mlir/test/lib/Dialect/Test/TestToLLVMIRTranslation.cpp (+1-2) - (modified) mlir/test/lib/Dialect/Test/TestTraits.cpp (+1-1) - (modified) mlir/test/lib/IR/TestBytecodeRoundtrip.cpp (+1-2) - (modified) mlir/test/lib/IR/TestClone.cpp (+1-1) - (modified) mlir/test/lib/IR/TestLazyLoading.cpp (-1) - (modified) mlir/test/lib/IR/TestRegions.cpp (-1) - (modified) mlir/test/lib/IR/TestSideEffects.cpp (+1-1) - (modified) mlir/test/lib/IR/TestSymbolUses.cpp (+1-1) - (modified) mlir/test/lib/IR/TestTypes.cpp (-1) - (modified) mlir/test/lib/IR/TestVisitorsGeneric.cpp (+1-1) - (modified) mlir/test/lib/Pass/TestPassManager.cpp (+1-2) - (modified) mlir/test/lib/Tools/PDLL/TestPDLL.cpp (+1-2) - (modified) mlir/test/lib/Transforms/TestInlining.cpp (+1-2) - (modified) mlir/test/lib/Transforms/TestInliningCallback.cpp (+2-2) - (modified) mlir/test/lib/Transforms/TestMakeIsolatedFromAbove.cpp (+1-2) - (added) mlir/test/mlir-tblgen/op-partitions-error.td (+74) - (added) mlir/test/mlir-tblgen/op-partitions.td (+110) - (modified) mlir/tools/mlir-tblgen/DialectGen.cpp (+25) - (modified) mlir/tools/mlir-tblgen/OpDefinitionsGen.cpp (+67-3) - (modified) mlir/tools/mlir-tblgen/OpGenHelpers.cpp (+79) - (modified) mlir/tools/mlir-tblgen/OpGenHelpers.h (+24-2) - (modified) mlir/unittests/IR/AdaptorTest.cpp (+1-2) - (modified) mlir/unittests/IR/IRMapping.cpp (+1-2) - (modified) mlir/unittests/IR/InterfaceAttachmentTest.cpp (+1-2) - (modified) mlir/unittests/IR/InterfaceTest.cpp (+1-2) - (modified) mlir/unittests/IR/OperationSupportTest.cpp (+1-2) - (modified) mlir/unittests/IR/PatternMatchTest.cpp (+1-2) - (modified) mlir/unittests/IR/ValueTest.cpp (-2) - (modified) mlir/unittests/Interfaces/MemorySlotUtilsTest.cpp (-1) - (modified) mlir/unittests/TableGen/OpBuildGen.cpp (+1-2) ``````````diff diff --git a/mlir/docs/DefiningDialects/Operations.md b/mlir/docs/DefiningDialects/Operations.md index f991f5308c5ae4..7c0228772d2488 100644 --- a/mlir/docs/DefiningDialects/Operations.md +++ b/mlir/docs/DefiningDialects/Operations.md @@ -1370,6 +1370,110 @@ cc_library( ) ``` +#### Natural Sharding with Op Partitions + +Sharding by index splits the ops arbitrarily: which shard an op ends up in +depends on the number of shards and on the names of all other ops. As an +alternative, a dialect can shard its ops "naturally" by grouping them into +named *op partitions*, such as arithmetic, memory or control flow ops. Each +partition is generated into its own declaration and definition files, almost as +if it were a dialect of its own, but all ops are still registered in one +dialect. + +Partitions are `Partition` records with a name, which must be a C++ +identifier. The dialect lists its partitions, and every op sets its `partition` +to one of these records (a different record with the same name is rejected), +typically with a `let` around a group of ops: + +```tablegen +def MyDialect_ArithPartition : Partition<"Arith">; +def MyDialect_MemoryPartition : Partition<"Memory">; + +def MyDialect : Dialect { + let name = "my_dialect"; + let opPartitions = [MyDialect_ArithPartition, MyDialect_MemoryPartition]; +} + +let partition = MyDialect_ArithPartition in { +def AddOp : MyDialect_Op<"add"> { ... } +def MulOp : MyDialect_Op<"mul"> { ... } +} + +def LoadOp : MyDialect_Op<"load"> { + let partition = MyDialect_MemoryPartition; + ... +} +``` + +A dialect can also name one of its partitions as `defaultOpPartition`. Ops that +do not set `partition` are then in this partition. This allows partitioning a +dialect incrementally: initially all ops are in the default partition, and +groups of ops can move to their own partitions one at a time: + +```tablegen +def MyDialect : Dialect { + let name = "my_dialect"; + let opPartitions = [MyDialect_CorePartition, MyDialect_MemoryPartition]; + let defaultOpPartition = MyDialect_CorePartition; +} + +// In partition `Core`. +def AddOp : MyDialect_Op<"add"> { ... } +``` + +Passing `-op-partition=<partition>` with a partition name to `-gen-op-decls` and +`-gen-op-defs` generates only the ops of that partition. + +Without `-op-partition`, `-gen-op-defs` generates all ops as before and +additionally defines the registration hooks of all partitions in +`GET_OP_CLASSES`. The hooks are private members of the dialect class, and the +dialect registers the ops of all partitions by calling `registerOperations()` +in its `initialize()`. A dialect can therefore declare its partitions and +register its ops with `registerOperations()` first, and split its generated +files by partition later. + +`mlir-tblgen` reports an error if a requested partition does not exist. A +partition for which no definitions are compiled results in a link error for its +hook. + +In CMake, generate and include the files of each partition: + +```cmake +set(LLVM_TARGET_DEFINITIONS MyDialectOps.td) +foreach(partition Arith Memory) + mlir_tablegen(MyDialect${partition}Ops.h.inc -gen-op-decls -op-partition=${partition}) + mlir_tablegen(MyDialect${partition}Ops.cpp.inc -gen-op-defs -op-partition=${partition}) +endforeach() +``` + +Since the generated declarations of a partition forward declare the ops of all +other partitions, each partition can get its own header, so that users only +include the declarations of the ops they use: + +```c++ +// MyDialectArithOps.h +#include "MyDialectBase.h" // The dialect, its attributes and types. + +#define GET_OP_CLASSES +#include "MyDialectArithOps.h.inc" + +// MyDialect.h, for users that want all ops. +#include "MyDialectArithOps.h" +#include "MyDialectMemoryOps.h" + +// MyDialectArithOps.cpp, a natural home for the C++ implementations of the +// arithmetic ops as well. +#include "MyDialectArithOps.h" +#include "OpUtils.h" + +#define GET_OP_CLASSES +#include "MyDialectArithOps.cpp.inc" +``` + +As with index sharding, utilities shared by the generated op definitions must +be declared in a header shared by the compilation units. A dialect with op +partitions cannot use `-op-shard-count`. + ## Constraints Constraint is a core concept in table-driven operation definition: operation diff --git a/mlir/include/mlir/IR/DialectBase.td b/mlir/include/mlir/IR/DialectBase.td index da669c6ada639f..92b6601b7e2877 100644 --- a/mlir/include/mlir/IR/DialectBase.td +++ b/mlir/include/mlir/IR/DialectBase.td @@ -19,6 +19,17 @@ include "mlir/IR/Utils.td" // Dialect definitions //===----------------------------------------------------------------------===// +// A partition of the ops of a dialect, for "natural sharding" of the generated +// op declarations and definitions. See `Dialect::opPartitions`. +class Partition<string name> { + // The name of the partition, a C++ identifier. `-gen-op-decls` and + // `-gen-op-defs` select a partition by this name with `-op-partition=...`. + string partitionName = name; + + assert !match(partitionName, "^[A-Za-z_][A-Za-z0-9_]*$"), + "op partition name '" # partitionName # "' is not a C++ identifier"; +} + class Dialect { // The name of the dialect. string name = ?; @@ -99,6 +110,28 @@ class Dialect { // If this dialect can be extended at runtime with new operations or types. bit isExtensible = 0; + + // The op partitions of this dialect, enabling "natural sharding" of its op + // declarations and definitions (see `Op::partition`). When non-empty, every + // op of the dialect must be in one of these partitions. + list<Partition> opPartitions = []; + + // Optional. The partition of the ops of this dialect that do not set their + // `partition`. Must be one of `opPartitions`. This allows partitioning the + // ops of a dialect incrementally: all ops start in the default partition and + // groups of ops can move to other partitions one at a time. + Partition defaultOpPartition = ?; + + assert !if(!initialized(defaultOpPartition), + !not(!empty(!filter(p, opPartitions, + !eq(p, defaultOpPartition)))), + 1), + "default op partition '" # + !if(!initialized(defaultOpPartition), + defaultOpPartition.partitionName # "' (record '" # + !cast<string>(defaultOpPartition), "") # + "') of dialect '" # name # + "' is not one of its `opPartitions`"; } #endif // DIALECTBASE_TD diff --git a/mlir/include/mlir/IR/OpBase.td b/mlir/include/mlir/IR/OpBase.td index a130d5847a1c72..3357f83e922bba 100644 --- a/mlir/include/mlir/IR/OpBase.td +++ b/mlir/include/mlir/IR/OpBase.td @@ -351,6 +351,30 @@ class Op<Dialect dialect, string mnemonic, list<Trait> props = []> { // Optional. The group of ops this op is part of. OpDocGroup opDocGroup = ?; + // Optional. The partition of this op. A dialect that lists its partitions in + // `opPartitions` is "naturally sharded": every op of the dialect must be in + // one of the listed partitions, `-gen-op-decls`/`-gen-op-defs` can generate + // the ops of one partition at a time with `-op-partition=...` into separate + // files. + // Defaults to the dialect's `defaultOpPartition`. The partition must be one + // of the `opPartitions` records of the dialect. + Partition partition = dialect.defaultOpPartition; + + assert !if(!initialized(partition), + !not(!empty(!filter(p, opDialect.opPartitions, + !eq(p, partition)))), + 1), + "op '" # NAME # "' is in partition '" # + !if(!initialized(partition), + partition.partitionName # "' (record '" # + !cast<string>(partition), "") # + "'), which is not one of the `opPartitions` of dialect '" # + opDialect.name # "'"; + + assert !or(!initialized(partition), !empty(opDialect.opPartitions)), + "op '" # NAME # "' is in no partition, but dialect '" # + opDialect.name # "' has `opPartitions` and no `defaultOpPartition`"; + // Dag containing the arguments of the op. Default to 0 arguments. dag arguments = (ins); diff --git a/mlir/include/mlir/TableGen/Dialect.h b/mlir/include/mlir/TableGen/Dialect.h index 68230196b17cc0..7913360c294e48 100644 --- a/mlir/include/mlir/TableGen/Dialect.h +++ b/mlir/include/mlir/TableGen/Dialect.h @@ -94,6 +94,10 @@ class Dialect { const llvm::DagInit *getDiscardableAttributes() const; + /// Returns the names of the op partitions of this dialect, see + /// `Op::partition`. A dialect with op partitions is "naturally sharded". + ArrayRef<StringRef> getOpPartitions() const; + const llvm::Record *getDef() const { return def; } // Returns whether two dialects are equal by checking the equality of the @@ -112,6 +116,7 @@ class Dialect { private: const llvm::Record *def; std::vector<StringRef> dependentDialects; + std::vector<StringRef> opPartitions; }; } // namespace tblgen } // namespace mlir diff --git a/mlir/include/mlir/TableGen/Operator.h b/mlir/include/mlir/TableGen/Operator.h index bb36c077708e92..cd321dcd6ca6f3 100644 --- a/mlir/include/mlir/TableGen/Operator.h +++ b/mlir/include/mlir/TableGen/Operator.h @@ -330,6 +330,9 @@ class Operator { /// Returns the dialect of the op. const Dialect &getDialect() const { return dialect; } + /// Returns the name of the op's partition, or an empty string if it has none. + StringRef getPartitionName() const; + /// Prints the contents in this operator to the given `os`. This is used for /// debugging purposes. void print(llvm::raw_ostream &os) const; diff --git a/mlir/lib/TableGen/Dialect.cpp b/mlir/lib/TableGen/Dialect.cpp index 84530bfa4f6fd2..a9f9b49b663ead 100644 --- a/mlir/lib/TableGen/Dialect.cpp +++ b/mlir/lib/TableGen/Dialect.cpp @@ -21,6 +21,9 @@ Dialect::Dialect(const llvm::Record *def) : def(def) { return; for (StringRef dialect : def->getValueAsListOfStrings("dependentDialects")) dependentDialects.push_back(dialect); + for (const llvm::Record *partition : + def->getValueAsListOfDefs("opPartitions")) + opPartitions.push_back(partition->getValueAsString("partitionName")); } StringRef Dialect::getName() const { return def->getValueAsString("name"); } @@ -110,6 +113,8 @@ const llvm::DagInit *Dialect::getDiscardableAttributes() const { return def->getValueAsDag("discardableAttrs"); } +ArrayRef<StringRef> Dialect::getOpPartitions() const { return opPartitions; } + bool Dialect::operator==(const Dialect &other) const { return def == other.def; } diff --git a/mlir/lib/TableGen/Operator.cpp b/mlir/lib/TableGen/Operator.cpp index f01820bcf411c3..eb9bdb1b2b5e70 100644 --- a/mlir/lib/TableGen/Operator.cpp +++ b/mlir/lib/TableGen/Operator.cpp @@ -151,6 +151,11 @@ void Operator::assertInvariants() const { StringRef Operator::getDialectName() const { return dialect.getName(); } +StringRef Operator::getPartitionName() const { + const llvm::Record *partition = def.getValueAsOptionalDef("partition"); + return partition ? partition->getValueAsString("partitionName") : ""; +} + StringRef Operator::getCppClassName() const { return cppClassName; } std::string Operator::getQualCppClassName() const { diff --git a/mlir/test/lib/Analysis/DataFlow/TestDenseBackwardDataFlowAnalysis.cpp b/mlir/test/lib/Analysis/DataFlow/TestDenseBackwardDataFlowAnalysis.cpp index f969869b924824..5bb7fbaaac6815 100644 --- a/mlir/test/lib/Analysis/DataFlow/TestDenseBackwardDataFlowAnalysis.cpp +++ b/mlir/test/lib/Analysis/DataFlow/TestDenseBackwardDataFlowAnalysis.cpp @@ -11,7 +11,7 @@ //===----------------------------------------------------------------------===// #include "TestDenseDataFlowAnalysis.h" -#include "TestOps.h" +#include "TestOpsInterfaces.h" #include "mlir/Analysis/DataFlow/DenseAnalysis.h" #include "mlir/Analysis/DataFlow/Utils.h" #include "mlir/Analysis/DataFlowFramework.h" diff --git a/mlir/test/lib/Analysis/DataFlow/TestDenseForwardDataFlowAnalysis.cpp b/mlir/test/lib/Analysis/DataFlow/TestDenseForwardDataFlowAnalysis.cpp index c8b67b4ba61c00..725daef9d08e3e 100644 --- a/mlir/test/lib/Analysis/DataFlow/TestDenseForwardDataFlowAnalysis.cpp +++ b/mlir/test/lib/Analysis/DataFlow/TestDenseForwardDataFlowAnalysis.cpp @@ -11,8 +11,7 @@ //===----------------------------------------------------------------------===// #include "TestDenseDataFlowAnalysis.h" -#include "TestDialect.h" -#include "TestOps.h" +#include "TestOpsInterfaces.h" #include "mlir/Analysis/DataFlow/DenseAnalysis.h" #include "mlir/Analysis/DataFlow/Utils.h" #include "mlir/Interfaces/SideEffectInterfaces.h" diff --git a/mlir/test/lib/Conversion/FuncToLLVM/TestConvertCallOp.cpp b/mlir/test/lib/Conversion/FuncToLLVM/TestConvertCallOp.cpp index 3b29867cc32631..89e02814950fd8 100644 --- a/mlir/test/lib/Conversion/FuncToLLVM/TestConvertCallOp.cpp +++ b/mlir/test/lib/Conversion/FuncToLLVM/TestConvertCallOp.cpp @@ -6,8 +6,7 @@ // //===----------------------------------------------------------------------===// -#include "TestDialect.h" -#include "TestOps.h" +#include "TestOpsConversion.h" #include "TestTypes.h" #include "mlir/Conversion/FuncToLLVM/ConvertFuncToLLVM.h" #include "mlir/Conversion/LLVMCommon/Pattern.h" diff --git a/mlir/test/lib/Dialect/Affine/TestReifyValueBounds.cpp b/mlir/test/lib/Dialect/Affine/TestReifyValueBounds.cpp index f11c14bc09fb66..efd8ee7a8ebf3f 100644 --- a/mlir/test/lib/Dialect/Affine/TestReifyValueBounds.cpp +++ b/mlir/test/lib/Dialect/Affine/TestReifyValueBounds.cpp @@ -6,8 +6,7 @@ // //===----------------------------------------------------------------------===// -#include "TestDialect.h" -#include "TestOps.h" +#include "TestOpsInterfaces.h" #include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/ValueBoundsOpInterfaceImpl.h" #include "mlir/Dialect/Affine/Transforms/Transforms.h" diff --git a/mlir/test/lib/Dialect/Bufferization/TestOneShotModuleBufferize.cpp b/mlir/test/lib/Dialect/Bufferization/TestOneShotModuleBufferize.cpp index d9f50ba0ee1ff4..634469278fa847 100644 --- a/mlir/test/lib/Dialect/Bufferization/TestOneShotModuleBufferize.cpp +++ b/mlir/test/lib/Dialect/Bufferization/TestOneShotModuleBufferize.cpp @@ -15,8 +15,7 @@ #include "mlir/Pass/Pass.h" #include "TestAttributes.h" // TestTensorEncodingAttr, TestMemRefLayoutAttr -#include "TestDialect.h" -#include "TestOps.h" +#include "TestOpsInterfaces.h" #include "llvm/ADT/TypeSwitch.h" diff --git a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp index 9fb89d3b4528b0..d0ca7fa5c2506f 100644 --- a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp +++ b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp @@ -6,7 +6,7 @@ // //===----------------------------------------------------------------------===// -#include "TestOps.h" +#include "TestOpsInterfaces.h" #include "mlir/Analysis/DataLayoutAnalysis.h" #include "mlir/Dialect/DLTI/DLTI.h" #include "mlir/IR/BuiltinAttributes.h" diff --git a/mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp b/mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp index c67bcd94ea0bd6..0f789a8c822e21 100644 --- a/mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp +++ b/mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp @@ -6,8 +6,7 @@ // //===----------------------------------------------------------------------===// -#include "TestDialect.h" -#include "TestOps.h" +#include "TestOpsInterfaces.h" #include "mlir/Dialect/Func/IR/FuncOps.h" #include "mlir/Dialect/Func/Transforms/FuncConversions.h" #include "mlir/IR/Builders.h" diff --git a/mlir/test/lib/Dialect/Test/CMakeLists.txt b/mlir/test/lib/Dialect/Test/CMakeLists.txt index ca0024b8434e16..d2dab9e88304fd 100644 --- a/mlir/test/lib/Dialect/Test/CMakeLists.txt +++ b/mlir/test/lib/Dialect/Test/CMakeLists.txt @@ -34,15 +34,18 @@ set(LLVM_TARGET_DEFINITIONS TestOps.td) mlir_tablegen(TestOpsDialect.h.inc -gen-dialect-decls -dialect=test) mlir_tablegen(TestOpsDialect.cpp.inc -gen-dialect-defs -dialect=test) mlir_tablegen(TestPatterns.inc -gen-rewriters) +# The op partitions of TestOps.td, each with its own header and source file. +foreach(partition Core Attributes Traits Patterns Conversion Interfaces Properties) + mlir_tablegen(TestOps${partition}.h.inc -gen-op-decls -op-partition=${partition}) + mlir_tablegen(TestOps${partition}.cpp.inc -gen-op-defs -op-partition=${partition}) +endforeach() add_public_tablegen_target(MLIRTestOpsIncGen) set(LLVM_TARGET_DEFINITIONS TestOpsSyntax.td) -mlir_tablegen(TestOpsSyntax.h.inc -gen-op-decls) -mlir_tablegen(TestOpsSyntax.cpp.inc -gen-op-defs) +mlir_tablegen(TestOpsSyntax.h.inc -gen-op-decls -op-partition=Syntax) +mlir_tablegen(TestOpsSyntax.cpp.inc -gen-op-defs -op-partition=Syntax) add_public_tablegen_target(MLIRTestOpsSyntaxIncGen) -add_sharded_ops(TestOps 20) - # Exclude tests from libMLIR.so add_mlir_library(MLIRTestDialect TestAttributes.cpp @@ -50,13 +53,18 @@ add_mlir_library(MLIRTestDialect TestFormatUtils.cpp TestInterfaces.cpp TestOpDefs.cpp - TestOps.cpp + TestOpsAttributes.cpp + TestOpsConversion.cpp + TestOpsCore.cpp + TestOpsInterfaces.cpp + TestOpsPatterns.cpp + TestOpsProperties.cpp + TestOpsTraits.cpp TestPatterns.cpp TestTraits.cpp TestTypes.cpp TestOpsSyntax.cpp TestDialectInterfaces.cpp - ${SHARDED_SRCS} EXCLUDE_FROM_LIBMLIR @@ -67,7 +75,6 @@ add_mlir_library(MLIRTestDialect MLIRTestTypeDefIncGen MLIRTestOpsIncGen MLIRTestOpsSyntaxIncGen - MLIRTestOpsShardGen MLIRConvertToEmitCPatternInterfaceIncGen ) mlir_target_link_libraries(MLIRTestDialect PUBLIC diff --git a/mlir/test/lib/Dialect/Test/TestDialect.cpp b/mlir/test/lib/Dialect/Test/TestDialect.cpp index 57bba2e5749a86..657f25771a45ee 100644 --- a/mlir/test/lib/Dialect/Test/TestDialect.cpp +++ b/mlir/test/lib/Dialect/Test/TestDialect.cpp @@ -346,9 +346,8 @@ struct TestOpEffectInterfaceFallback void TestDialect::initialize() { registerAttributes(); registerTypes(); - registerOpsSyntax(); addOperations<ManualCppOpWithFold>(); - registerTestDialectOperations(this); + registerOperations(); registerDynamicOp(getDynamicGenericOp(this)); registerDynamicOp(getDynamicTerminatorOp(this)); registerDynamicOp(getDynamicNoTerminatorOp(this)); diff --git a/mlir/test/lib/Dialect/Test/TestDialect.td b/mlir/test/lib/Dialect/Test/TestDialect.td index 37a263f1d10b8b..debb239addc056 100644 --- a/mlir/test/lib/Dialect/Test/TestDialect.td +++ b/mlir/test/lib/Dialect/Test/TestDialect.td @@ -11,6 +11,19 @@ include "mlir/IR/OpBase.td" +// Op partitions, each generated and compiled separately. Ops from TestOps.td +// are in `Core` unless a `let` block or their base class picks another +// partition. All ops from TestOpsSyntax.td are in `Syntax`, which is generated +// from that file. +def Test_CorePartition : Partition<"Core">; +def Test_AttributesPartition : Partition<"Attributes">; +def Test_TraitsPartition : Partition<"Traits">; +def Test_PatternsPartition : Partition<"Patterns">; +def Test_ConversionPartition : Partition<"Conversion">; +def Test_InterfacesPartition : Partition<"Interfaces">; +def Test_PropertiesPartition : Partition<"Properties">; +def Test_SyntaxPartition : Partition<"Syntax">; + def Test_Dialect : Dialect { let name = "test"; let cppNamespace = "::test"; @@ -24,6 +37,12 @@ def Test_Dialect : Dialect { let useDefaultTypePrinterParser = 0; let useDefaultAttributePrinterParser = 1; let isExtensible = 1; + let opPartitions = [ + Test_CorePartition, Test_AttributesPartition, Test_TraitsPartition, + Test_PatternsPartition, Test_ConversionPartition, Test_InterfacesPartition, + Test_PropertiesPartition, Test_SyntaxPartition + ]; + let defaultOpPartition = Test_CorePartition; let dependentDialects = [ "::mlir::DLTIDialect", "::mlir::bufferization::BufferizationDialect" @@ -37,7 +56,6 @@ def Test_Dialect : Dialect { void registerAttributes(); void registerInterfaces(); void registerTypes(); - void registerOpsSyntax(); // Provides a custom printing/parsing for some operations. ::std::optional<ParseOpHook> diff --git a/mlir/test/lib/Dialect/Test/TestFromLLVMIRTranslation.cpp b/mlir/test/lib/Dialect/Test/TestFromLLVMIRTranslation.cpp index b98f6ce32cdceb..c1f8629c65df42 100644 --- a/mlir/test/lib/Dialect/Test/TestFromLLVMI... [truncated] `````````` </details> https://github.com/llvm/llvm-project/pull/239254 _______________________________________________ llvm-branch-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
