Mousius commented on code in PR #71: URL: https://github.com/apache/tvm-rfcs/pull/71#discussion_r873834623
########## rfcs/0070-target-preprocessing.md: ########## @@ -0,0 +1,208 @@ +- Feature Name: target-architecture-preprocessor +- Start Date: 2022-04-04 +- RFC PR: [apache/tvm-rfcs#0070](https://github.com/apache/tvm-rfcs/pull/0000) +- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000) + +# Summary +[summary]: #summary +Provide a standard and easily testable way to inspect architecture extensions and provide them to the various parts of TVM which utilise that information. + +# Motivation +[motivation]: #motivation +TVM has multiple ways to define a `Target`s architectural features for use in deciding on schedules or other calculations, here's a few different ways we do this: + +* CPU to Feature Mapping: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/python/tvm/target/arm_isa.py#L22-L39 +* Inspecting `Target` in utility functions: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/python/tvm/topi/arm_cpu/arm_utils.py#L24-L70 +* Inspecting `Target` in utility functions inside legalization code: https://github.com/apache/tvm/blob/02fbaf0ed9120a8f95155e63de42459f230584aa/python/tvm/relay/qnn/op/legalizations.py#L350-L359 +* Inspecting `Target` inside the definition a strategy: https://github.com/apache/tvm/blob/b542724873140bb051492530d97a78b9b7b7983d/python/tvm/relay/op/strategy/arm_cpu.py#L232 +* Processing bespoke Compiler arguments: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/src/relay/backend/contrib/cmsisnn/compiler_attrs.cc#L47-L70 +* Registered as a `PackedFunc` (https://github.com/apache/tvm/blob/24e5498021cecca2fe7d44149ce90efe28b6d930/python/tvm/topi/x86/utils.py#L21-L34) and then used as part of `Op` processing: https://github.com/apache/tvm/blob/24e5498021cecca2fe7d44149ce90efe28b6d930/src/relay/qnn/op/requantize_config.h#L58-L73 + +This RFC aims to standardise the way in which we convert `Target` attributes into architectural features by processing them ahead of time. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Two additional pre-processors can be added to the `Target`, for users to preprocess architectural information when the `Target` is created: +* Architecture Pre-processing - maps `Target` `attrs` to a new `arch` object +* Keys Pre-processing - maps `Target` `attrs` and `keys` to a new set of `keys` + +These new preprocessors will be illustrated using examples targeting TVM for Arm(R) Cortex(R)-M4. + +## Architecture Pre-processing +```c++ +Target("c") + .set_arch_preprocessor(MyArchPreprocessor) +``` + +This takes the `attrs` from `Target` and converts them into an object representing the architectural features of the `Target`, which can then be accessed using the `GetArch` method similar to `GetAttr`: + +```c++ +Target my_target("c -mcpu=cortex-m4"); +my_target->GetArch<Bool>("is_aarch64", false); // false +my_target->GetArch<Bool>("has_dsp", false); // true +``` + +```python +my_target = Target("c -mcpu=cortex-m4") +my_target.arch.is_aarch64 // false +my_target.arch.has_dsp // true +``` + +## Keys Pre-processing + +```c++ +Target("c") + .set_keys_preprocessor(MyKeysPreprocessor) +``` + +This takes the `attrs` from `Target` and maps them to relevant `keys` for use when selecting schedules: + +```c++ +Target my_target("c -mcpu=cortex-m4"); +my_target->keys; // ["arm_cpu", "cpu"] <-- "cpu" is taken from default keys and merged by the pre-preprocessor +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +Currently, there is a single `preprocessor` which takes an input of `attrs` and expects the same `attrs` returned with pre-processing applied: + +https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/src/target/target.cc#L810-L814 + +In extension to this, a series of new pre-processors will be defined: + +```c++ +using TargetAttrs = Map<String, ObjectRef>; +using TargetArch = Map<String, ObjectRef>; +using TargetKeys = Array<String>; + +using FTVMAttrPreprocessor = runtime::TypedPackedFunc<TargetAttrs(TargetAttrs)>; +using FTVMArchPreprocessor = runtime::TypedPackedFunc<TargetArch(TargetAttrs)>; +using FTVMKeysPreprocessor = runtime::TypedPackedFunc<TargetKeys(TargetAttrs, TargetKeys)>; +``` + +These implementations can be stored under `src/target/preprocessors/<arch_identifier>.{cc.h}` to allow them to be composed together such as: + +* src/target/preprocessors/aarch64.cc +* src/target/preprocessors/cpu.cc + +Where the `cpu` pre-processor can utilise the `aarch64` pre-processor if detected. + +## Rename Attr Preprocessor +To help avoid confusion between the existing `attrs` `preprocessor` and the new pre-processors, the `attrs` pre-processor will be renamed from `preprocessor` to `attr_preprocessor`: + +```c++ +class TargetKind { + ... + FTVMAttrPreprocessor attr_preprocessor; + + ... +} +``` + +## Architecture Preprocessor +The first new pre-processor, which processes `attrs` in to an `arch` object, is registered as a new field is added to `TargetKind`: + +```c++ +class TargetKind { + ... + FTVMArchPreprocessor arch_preprocessor; Review Comment: > Clang driver takes flags given by the user and translates them into specific "subtarget" configuration. There is a flag -target <triple>, which roughly corresponds to a TargetMachine, and then there are other details (implicit or explicit) that eventually create TargetSubtargetInfo. Apologies, I oversimplified, you're right that Clang/LLVM has a few more data structures to describe this (including `SubTarget`s) - in this RFC we're essentially using `Map<String, ObjectRef>` as a replacement for most of the `SubTarget` in the `.arch` property. I can add further details to the Clang/LLVM Prior Art to reflect this 😸 > The target kind "llvm", on the other hand, does correspond to any generic CPU, be it x86 or Arm. Flags specific to x86 may not be applicable to Arm, and vice versa. If we attach every arch preprocessor to kind "llvm", then whatever code executes them will have to know which preprocessor to apply, and currently there is no information that would aid with this selection. Sorry, I'm not understanding so I'll provide an example 😸 , in the case of LLVM we'd attach a pre-processor which can inspect the attributes and do something similar to: ```c++ Map<String, ObjectRef> llvm_preprocessor(...attrs...) { bool is_aarch64 = ...logic ...; if (is_aarch64) { return aarch64_preprocessor(...attrs...); } } ``` Therefore, whenever you instantiate a `Target("llvm -march=aarch64-woof-woof-woof")` the preprocessor would give you a `target.arch` `Map` which would be pre-processed using the `aarch64` processor, similar to the snippet I pasted from `clang` with the `switch` logic. This does mean we have to create a preprocessor for both the `llvm` and `c` `Target`s to map `mcpu`, `march`, `mtune` or `mtriple` to the relevant logic path in the same way that LLVM suggests processing `march` and `mcpu` for `SubTargetInfo` - this preprocessor would have the freedom to generate any eventual `Map<String, ObjectRef>` for the specific details of a given architecture which is being targeted by the `llvm` `Target` of TVM. > This goes beyond the scope of this RFC, but my vote would be to eliminate "llvm" as a target kind, and replace it with specific kinds of CPUs. We could have target "host" that would correspond to the compilation host (and can be automatically translated to an appropriate specific CPU kind). The `Target` is definitely a difficult problem to unpick, so glad you agree it's out of scope 😸 , personally I would avoid magic `Target`s such as "composite" and "host" in favour of better defining them - `llvm` is maybe just a code generator for however we describe a CPU `Target`? Potentially worth starting an RFC to get the discussion going and then discuss in the community meetup? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
