junrushao1994 commented on a change in pull request #5:
URL: https://github.com/apache/tvm-rfcs/pull/5#discussion_r676888273
##########
File path: rfcs/0001-meta-schedule-autotensorir.md
##########
@@ -266,55 +322,83 @@ sch.reorder(l14, l18, l15, l19, l22, l16, l20, l23, l17,
l21)
### 4.2. Exploring the Design Space
Meta Schedule provides several built-in exploration strategies to exhaustively
or efficiently search
-for efficient schedules.
-
-**Random search by replaying schedule functions.** With a user-provided
schedule function
-as a black-box design space generator, our system could repetitively invoke
such an opaque function
-without doing any extra analysis. The function could be written in C++ or
Python, or any language
-that implements packed function FFI. If sampling instructions are present in
the function, then each
-invocation results in a different IRModule after being scheduled because the
random decisions are
-possibly changed across different runs. Effectively, it is equivalent to
-random exploration without trace, allowing the flexibility for users to define
arbitrary functions
+for efficient schedules. Those strategies are mostly supported by re-execute
either a function or a
+trace with a builtin interpreter in meta schedule, and this process is called
**replay**.
+
+#### Random search by replaying schedule functions
+
+With a user-provided schedule function
+as a black-box design space generator, meta schedule could repetitively
invokes such an opaque TVM
+packed function without doing any extra analysis.
+If sampling instructions are present in the trace, then scheduling is
non-deterministic
+(random decisions may not be repeated across runs)
+Effectively, it is equivalent to random exploration without trace,
+allowing the flexibility for users to define arbitrary functions
that trace may not well support (e.g. control flow divergence based on the
value of intermediate
random variables), but it forbids future opportunity of any trace-based
analysis.
-**Random search by replaying traces.** Traces are obtained from a design space
generator, and
-replayed with a builtin interpreter in our system. If sampling instructions
are present on the
-traces, then their random decisions are mutated during each replay, i.e. jumps
to a new point in the
+#### Random search by replaying traces
+
+A builtin interpreter directly replays the traces obtained
+from manual schedule, template-based or template-free design space generation.
+If sampling instructions are present on the traces,
+then their random decisions are mutated during each replay, i.e. jumps to a
new point in the
design space. Therefore, repetitive replay of those traces are equivalent to
exploration of the
-design space. Our system could potentially benefit from trace-based analysis,
including rejecting
-obviously invalid schedules (e.g. using too much CUDA resources), doing
dead-code elimination to
-simplify a trace, extracting trace-based features used in the cost model, etc.
+design space. meta schedule could potentially benefit from trace-based
analysis, making the search more
Review comment:
That makes sense. I also added feature extraction related stuff
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
