This is an automated email from the ASF dual-hosted git repository.
lmzheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/tvm-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 8828684 Build at Wed Mar 3 01:20:50 PST 2021
8828684 is described below
commit 88286848906663319587f02c11f37dd2fe696f30
Author: Lianmin Zheng <[email protected]>
AuthorDate: Wed Mar 3 01:20:50 2021 -0800
Build at Wed Mar 3 01:20:50 PST 2021
---
2017/08/17/tvm-release-announcement.html | 2 +-
...s-with-TVM-A-Depthwise-Convolution-Example.html | 2 +-
2017/10/06/nnvm-compiler-announcement.html | 2 +-
...s-to-TVM-Stack-and-NNVM-Compiler-with-ROCm.html | 2 +-
2017/11/08/android-rpc-introduction.html | 2 +-
2018/01/16/opt-mali-gpu.html | 2 +-
2018/03/12/webgl.html | 2 +-
2018/03/23/nmt-transformer-optimize.html | 2 +-
2018/07/12/vta-release-announcement.html | 2 +-
2018/08/10/DLPack-Bridge.html | 2 +-
2018/10/03/auto-opt-all.html | 2 +-
2018/10/09/ml-in-tees.html | 2 +-
2018/12/18/lowprecision-conv.html | 2 +-
2019/01/19/Golang.html | 2 +-
2019/03/18/tvm-apache-announcement.html | 2 +-
2019/04/29/opt-cuda-quantized.html | 2 +-
2019/05/30/pytorch-frontend.html | 2 +-
...machine-learning-to-webassembly-and-webgpu.html | 2 +-
2020/06/04/tinyml-how-tvm-is-taming-tiny.html | 2 +-
2020/07/14/bert-pytorch-tvm.html | 2 +-
.../15/how-to-bring-your-own-codegen-to-tvm.html | 2 +-
2020/09/26/bring-your-own-datatypes.html | 2 +-
2021/03/03/intro-auto-scheduler.html | 321 +++++++++++++++++++++
atom.xml | 253 +++++++++-------
blog.html | 10 +
community.html | 4 +
feed.xml | 291 +++++++++----------
images/community/sjtu.png | Bin 0 -> 236508 bytes
images/intro-auto-scheduler/code_perf.png | Bin 0 -> 36724 bytes
images/intro-auto-scheduler/search_overview.png | Bin 0 -> 433415 bytes
images/intro-auto-scheduler/search_time.png | Bin 0 -> 45583 bytes
images/intro-auto-scheduler/workflow.png | Bin 0 -> 1014076 bytes
rss.xml | 255 +++++++++-------
sitemap.txt | 1 +
34 files changed, 789 insertions(+), 390 deletions(-)
diff --git a/2017/08/17/tvm-release-announcement.html
b/2017/08/17/tvm-release-announcement.html
index ea95cf0..dbd65e1 100644
--- a/2017/08/17/tvm-release-announcement.html
+++ b/2017/08/17/tvm-release-announcement.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>TVM: An End to End IR Stack for Deploying Deep Learning Workloads on
Hardware Platforms </h1>
<p class="post-meta">
- <time datetime="2017-08-17T15:00:00-04:00" itemprop="datePublished">
+ <time datetime="2017-08-17T12:00:00-07:00" itemprop="datePublished">
Aug 17, 2017
</time>
diff --git
a/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html
b/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html
index 96b2e16..13a15a3 100644
---
a/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html
+++
b/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Optimize Deep Learning GPU Operators with TVM: A Depthwise
Convolution Example </h1>
<p class="post-meta">
- <time datetime="2017-08-22T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2017-08-22T00:00:00-07:00" itemprop="datePublished">
Aug 22, 2017
</time>
diff --git a/2017/10/06/nnvm-compiler-announcement.html
b/2017/10/06/nnvm-compiler-announcement.html
index 40557e0..b627ca6 100644
--- a/2017/10/06/nnvm-compiler-announcement.html
+++ b/2017/10/06/nnvm-compiler-announcement.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>NNVM Compiler: Open Compiler for AI Frameworks </h1>
<p class="post-meta">
- <time datetime="2017-10-06T11:30:00-04:00" itemprop="datePublished">
+ <time datetime="2017-10-06T08:30:00-07:00" itemprop="datePublished">
Oct 6, 2017
</time>
diff --git
a/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm.html
b/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm.html
index 06f20bd..e6a6c2f 100644
--- a/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm.html
+++ b/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Bringing AMDGPUs to TVM Stack and NNVM Compiler with ROCm </h1>
<p class="post-meta">
- <time datetime="2017-10-30T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2017-10-30T00:00:00-07:00" itemprop="datePublished">
Oct 30, 2017
</time>
diff --git a/2017/11/08/android-rpc-introduction.html
b/2017/11/08/android-rpc-introduction.html
index 7d15d82..f7e34b5 100644
--- a/2017/11/08/android-rpc-introduction.html
+++ b/2017/11/08/android-rpc-introduction.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Remote Profile and Test Deep Learning Cross Compilation on Mobile
Phones with TVM RPC </h1>
<p class="post-meta">
- <time datetime="2017-11-08T00:00:00-05:00" itemprop="datePublished">
+ <time datetime="2017-11-08T00:00:00-08:00" itemprop="datePublished">
Nov 8, 2017
</time>
diff --git a/2018/01/16/opt-mali-gpu.html b/2018/01/16/opt-mali-gpu.html
index a039779..40fc7f0 100644
--- a/2018/01/16/opt-mali-gpu.html
+++ b/2018/01/16/opt-mali-gpu.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Optimizing Mobile Deep Learning on ARM GPU with TVM </h1>
<p class="post-meta">
- <time datetime="2018-01-16T00:00:00-05:00" itemprop="datePublished">
+ <time datetime="2018-01-16T00:00:00-08:00" itemprop="datePublished">
Jan 16, 2018
</time>
diff --git a/2018/03/12/webgl.html b/2018/03/12/webgl.html
index 792c922..74313b5 100644
--- a/2018/03/12/webgl.html
+++ b/2018/03/12/webgl.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Compiling Deep Learning Models to WebGL with TVM </h1>
<p class="post-meta">
- <time datetime="2018-03-12T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2018-03-12T00:00:00-07:00" itemprop="datePublished">
Mar 12, 2018
</time>
diff --git a/2018/03/23/nmt-transformer-optimize.html
b/2018/03/23/nmt-transformer-optimize.html
index 2182327..35c211a 100644
--- a/2018/03/23/nmt-transformer-optimize.html
+++ b/2018/03/23/nmt-transformer-optimize.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Bringing TVM into TensorFlow for Optimizing Neural Machine
Translation on GPU </h1>
<p class="post-meta">
- <time datetime="2018-03-23T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2018-03-23T00:00:00-07:00" itemprop="datePublished">
Mar 23, 2018
</time>
diff --git a/2018/07/12/vta-release-announcement.html
b/2018/07/12/vta-release-announcement.html
index c60a3e1..1250749 100644
--- a/2018/07/12/vta-release-announcement.html
+++ b/2018/07/12/vta-release-announcement.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>VTA: An Open, Customizable Deep Learning Acceleration Stack </h1>
<p class="post-meta">
- <time datetime="2018-07-12T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2018-07-12T00:00:00-07:00" itemprop="datePublished">
Jul 12, 2018
</time>
diff --git a/2018/08/10/DLPack-Bridge.html b/2018/08/10/DLPack-Bridge.html
index 7ec1aaa..af4d193 100644
--- a/2018/08/10/DLPack-Bridge.html
+++ b/2018/08/10/DLPack-Bridge.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Building a Cross-Framework Deep Learning Compiler via DLPack </h1>
<p class="post-meta">
- <time datetime="2018-08-10T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2018-08-10T00:00:00-07:00" itemprop="datePublished">
Aug 10, 2018
</time>
diff --git a/2018/10/03/auto-opt-all.html b/2018/10/03/auto-opt-all.html
index 98269c7..ac36190 100644
--- a/2018/10/03/auto-opt-all.html
+++ b/2018/10/03/auto-opt-all.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Automatic Kernel Optimization for Deep Learning on All Hardware
Platforms </h1>
<p class="post-meta">
- <time datetime="2018-10-03T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2018-10-03T00:00:00-07:00" itemprop="datePublished">
Oct 3, 2018
</time>
diff --git a/2018/10/09/ml-in-tees.html b/2018/10/09/ml-in-tees.html
index 992e1a3..0f59a69 100644
--- a/2018/10/09/ml-in-tees.html
+++ b/2018/10/09/ml-in-tees.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Efficient Privacy-Preserving ML Using TVM </h1>
<p class="post-meta">
- <time datetime="2018-10-09T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2018-10-09T00:00:00-07:00" itemprop="datePublished">
Oct 9, 2018
</time>
diff --git a/2018/12/18/lowprecision-conv.html
b/2018/12/18/lowprecision-conv.html
index c5def47..f32251d 100644
--- a/2018/12/18/lowprecision-conv.html
+++ b/2018/12/18/lowprecision-conv.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Automating Generation of Low Precision Deep Learning Operators </h1>
<p class="post-meta">
- <time datetime="2018-12-18T00:00:00-05:00" itemprop="datePublished">
+ <time datetime="2018-12-18T00:00:00-08:00" itemprop="datePublished">
Dec 18, 2018
</time>
diff --git a/2019/01/19/Golang.html b/2019/01/19/Golang.html
index 27a39f0..6b8b94a 100644
--- a/2019/01/19/Golang.html
+++ b/2019/01/19/Golang.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>TVM Golang Runtime for Deep Learning Deployment </h1>
<p class="post-meta">
- <time datetime="2019-01-19T00:00:00-05:00" itemprop="datePublished">
+ <time datetime="2019-01-19T00:00:00-08:00" itemprop="datePublished">
Jan 19, 2019
</time>
diff --git a/2019/03/18/tvm-apache-announcement.html
b/2019/03/18/tvm-apache-announcement.html
index 386de84..19b5017 100644
--- a/2019/03/18/tvm-apache-announcement.html
+++ b/2019/03/18/tvm-apache-announcement.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>TVM Deep Learning Compiler Joins Apache Software Foundation </h1>
<p class="post-meta">
- <time datetime="2019-03-18T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2019-03-18T00:00:00-07:00" itemprop="datePublished">
Mar 18, 2019
</time>
diff --git a/2019/04/29/opt-cuda-quantized.html
b/2019/04/29/opt-cuda-quantized.html
index 3b401af..1c55a9a 100644
--- a/2019/04/29/opt-cuda-quantized.html
+++ b/2019/04/29/opt-cuda-quantized.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Automating Optimization of Quantized Deep Learning Models on CUDA
</h1>
<p class="post-meta">
- <time datetime="2019-04-29T12:00:00-04:00" itemprop="datePublished">
+ <time datetime="2019-04-29T09:00:00-07:00" itemprop="datePublished">
Apr 29, 2019
</time>
diff --git a/2019/05/30/pytorch-frontend.html b/2019/05/30/pytorch-frontend.html
index ad8281b..a4dd9a3 100644
--- a/2019/05/30/pytorch-frontend.html
+++ b/2019/05/30/pytorch-frontend.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Integrating TVM into PyTorch </h1>
<p class="post-meta">
- <time datetime="2019-05-30T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2019-05-30T00:00:00-07:00" itemprop="datePublished">
May 30, 2019
</time>
diff --git
a/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu.html
b/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu.html
index 38bd956..50f01e7 100644
--- a/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu.html
+++ b/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Compiling Machine Learning to WASM and WebGPU with Apache TVM </h1>
<p class="post-meta">
- <time datetime="2020-05-14T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2020-05-14T00:00:00-07:00" itemprop="datePublished">
May 14, 2020
</time>
diff --git a/2020/06/04/tinyml-how-tvm-is-taming-tiny.html
b/2020/06/04/tinyml-how-tvm-is-taming-tiny.html
index bcb1aed..ec640c7 100644
--- a/2020/06/04/tinyml-how-tvm-is-taming-tiny.html
+++ b/2020/06/04/tinyml-how-tvm-is-taming-tiny.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>TinyML - How TVM is Taming Tiny </h1>
<p class="post-meta">
- <time datetime="2020-06-04T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2020-06-04T00:00:00-07:00" itemprop="datePublished">
Jun 4, 2020
</time>
diff --git a/2020/07/14/bert-pytorch-tvm.html b/2020/07/14/bert-pytorch-tvm.html
index a563504..387e219 100644
--- a/2020/07/14/bert-pytorch-tvm.html
+++ b/2020/07/14/bert-pytorch-tvm.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Bridging PyTorch and TVM </h1>
<p class="post-meta">
- <time datetime="2020-07-14T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2020-07-14T00:00:00-07:00" itemprop="datePublished">
Jul 14, 2020
</time>
diff --git a/2020/07/15/how-to-bring-your-own-codegen-to-tvm.html
b/2020/07/15/how-to-bring-your-own-codegen-to-tvm.html
index a2066ec..3d39e96 100644
--- a/2020/07/15/how-to-bring-your-own-codegen-to-tvm.html
+++ b/2020/07/15/how-to-bring-your-own-codegen-to-tvm.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>How to Bring Your Own Codegen to TVM </h1>
<p class="post-meta">
- <time datetime="2020-07-15T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2020-07-15T00:00:00-07:00" itemprop="datePublished">
Jul 15, 2020
</time>
diff --git a/2020/09/26/bring-your-own-datatypes.html
b/2020/09/26/bring-your-own-datatypes.html
index 0dc4fb0..135d0db 100644
--- a/2020/09/26/bring-your-own-datatypes.html
+++ b/2020/09/26/bring-your-own-datatypes.html
@@ -140,7 +140,7 @@
<div class="span14 w-100">
<h1>Bring Your Own Datatypes: Enabling Custom Datatype Exploration in
TVM </h1>
<p class="post-meta">
- <time datetime="2020-09-26T00:00:00-04:00" itemprop="datePublished">
+ <time datetime="2020-09-26T00:00:00-07:00" itemprop="datePublished">
Sep 26, 2020
</time>
diff --git a/2021/03/03/intro-auto-scheduler.html
b/2021/03/03/intro-auto-scheduler.html
new file mode 100644
index 0000000..e10a971
--- /dev/null
+++ b/2021/03/03/intro-auto-scheduler.html
@@ -0,0 +1,321 @@
+<html lang="en">
+<head>
+ <meta charset="UTF-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <title>Introducing TVM Auto-scheduler (a.k.a. Ansor)</title>
+ <link rel="shortcut icon" href="/assets/images/favicon.ico">
+ <link rel="stylesheet"
href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css";
integrity="sha384-MCw98/SFnGE8fJT3GXwEOngsV7Zt27NXFoaoApmYm81iuXoPkFOJwJ8ERdknLPMO"
crossorigin="anonymous">
+ <link rel="stylesheet" href="/css/slick.css">
+ <link rel="stylesheet" href="/css/slick-theme.css">
+ <link rel="stylesheet" href="/css/custom.css">
+</head>
+<body>
+
+
+<div class="bannerPage">
+ <header class="header">
+ <div class="container">
+ <div class="headerInner d-flex justify-content-between
align-items-center">
+ <div class="headerLogo">
+ <a href="/"><img src="/assets/images/logo.svg" alt="Logo"></a>
+ </div>
+ <div id="headMenu" class="headerNav">
+ <button type="button" id="closeHeadMenu" class="navCloseBtn"><img
src="/assets/images/close-icon.svg"
+ alt="Close"></button>
+ <ul class="nav">
+
+ <li class="nav-item">
+ <a class="nav-link" href="/community">Community</a>
+ </li>
+
+ <li class="nav-item">
+ <a class="nav-link" href="/download">Download</a>
+ </li>
+
+ <li class="nav-item">
+ <a class="nav-link" href="/vta">VTA</a>
+ </li>
+
+ <li class="nav-item">
+ <a class="nav-link" href="/blog">Blog</a>
+ </li>
+
+ <li class="nav-item">
+ <a class="nav-link" href="https://tvm.apache.org/docs/";>Docs</a>
+ </li>
+
+ <li class="nav-item">
+ <a class="nav-link" href="https://tvmconf.org/";>Conference</a>
+ </li>
+
+ <li class="nav-item">
+ <a class="nav-link"
href="https://github.com/apache/incubator-tvm/";>Github</a>
+ </li>
+
+</ul>
+ <div class="responsiveasfdropdown">
+ <button type="button" class="btn-link">
+ ASF
+ </button>
+ <ul>
+
+ <li>
+ <a href="https://www.apache.org/";>Apache Homepage</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/licenses/";>License</a>
+ </li>
+
+ <li>
+ <a
href="https://www.apache.org/foundation/sponsorship.html";>Sponsorship</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/security/";>Security</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/foundation/thanks.html";>Thanks</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/events/current-event";>Events</a>
+ </li>
+
+</ul>
+ </div>
+ </div>
+ <div class="responsiveMenuIcon">
+ <button type="button" id="menuBtn" class="btn-menu"><img
src="/assets/images/menu-icon.svg"
+ alt="Menu Icon" /></button>
+ </div>
+ <div class="asfDropdown">
+ <div class="dropdown">
+ <button type="button" class="btn-link dropdown-toggle"
data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ ASF
+ </button>
+ <div class="dropdown-menu dropdown-menu-right">
+ <ul>
+
+ <li>
+ <a href="https://www.apache.org/";>Apache Homepage</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/licenses/";>License</a>
+ </li>
+
+ <li>
+ <a
href="https://www.apache.org/foundation/sponsorship.html";>Sponsorship</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/security/";>Security</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/foundation/thanks.html";>Thanks</a>
+ </li>
+
+ <li>
+ <a href="https://www.apache.org/events/current-event";>Events</a>
+ </li>
+
+</ul>
+ </div>
+ </div>
+ </div>
+ </div>
+ </div>
+ </header>
+
+</div>
+
+
+<div class="container">
+<div class="content">
+ <div class="row">
+ <div class="span14 w-100">
+ <h1>Introducing TVM Auto-scheduler (a.k.a. Ansor) </h1>
+ <p class="post-meta">
+ <time datetime="2021-03-03T00:00:00-08:00" itemprop="datePublished">
+ Mar 3, 2021
+ </time>
+
+ • <span itemprop="author" itemscope
itemtype="http://schema.org/Person";>
+ <span itemprop="name">Lianmin Zheng, Chengfan Jia, Minmin Sun, Zhao
Wu, Cody Hao Yu</span>
+ </span>
+
+ </p>
+ <p class="post-meta">
+ </p>
+ </br>
+ <p>Optimizing the execution speed of deep neural networks is extremely
hard with the growing
+model size, operator diversity, and hardware heterogeneity.
+From a computational perspective, deep neural networks are just layers and
layers of tensor computations.
+These tensor computations, such as matmul and conv2d, can be easily described
by mathematical expressions.
+However, providing high-performance implementations for them on modern
hardware can be very challenging.
+We have to apply various low-level optimizations and utilize special hardware
intrinsics to achieve high performance.
+It takes huge engineering effort to build linear algebra and neural network
acceleration libraries like CuBLAS, CuDNN, oneMKL, and oneDNN.</p>
+
+<p>Our life will be much easier if we can just write mathematical expressions
and have something
+magically turn them into efficient code implementations.
+Three years ago, deep learning compiler TVM and its search module AutoTVM were
built as the first step towards this goal.
+AutoTVM employs a template-based search algorithm to find efficient
implementations for a given tensor computation.
+However, it is a template-based approach, so it still requires domain experts
to implement a non-trivial manual template
+for every operator on every platform.
+Today, there are more than 15k lines of code for these templates in the TVM
code repository.
+Besides being very hard to develop, these templates often have inefficient and
limited search spaces,
+making them unable to achieve optimal performance.</p>
+
+<p>To address the limitations of AutoTVM, we started project Ansor aiming at a
fully automated auto-scheduler for
+generating code for tensor computations.
+Ansor auto-scheduler only takes tensor expressions as input and generates
high-performance code without manual templates.
+We made innovations in the search space construction and search algorithm.
+As a result, the auto-scheduler can achieve better performance with less
search time in a more automated way.</p>
+
+<p>Ansor auto-scheduler is now integrated into Apache TVM as <code
class="language-plaintext highlighter-rouge">tvm.auto_scheduler</code> package.
+This is a joint effort by collaborators from UC Berkeley, Alibaba, AWS and
OctoML.
+Detailed tutorials are available for Intel CPUs, ARM CPUs, NVIDIA CPUs, and
Mali GPUs on the TVM website [1].
+In this blog post, we will give a high-level introduction and show some
benchmark results.</p>
+
+<h1 id="system-overview">System Overview</h1>
+
+<h2 id="autotvm-vs-auto-scheduler">AutoTVM vs Auto-scheduler</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/workflow.png" alt="image" width="75%" /></p>
+<center> Table 1. Workflow Comparision </center>
+<p></p>
+
+<p>Table 1 compares the workflow for generating code for an operator in
AutoTVM and auto-scheduler.
+In AutoTVM, the developer has to go through three steps.
+In step 1, the developer has to write the compute definition in TVM’s tensor
expression language.
+This part is relatively easy because TVM’s tensor expression language looks
just like math expressions.
+In step 2, the developer has to write a schedule template, which typically
consists of 20-100 lines of tricky DSL code.
+This part requires domain expertise of both the target hardware architecture
and operator semantics, so it is difficult.
+The last step, step 3, is automated by a search algorithm.</p>
+
+<p>In auto-scheduler, we eliminate the most difficult step 2 by automatic
search space construction and accelerate step 3 with a better search algorithm.
+By doing automatic search space construction, we not only eliminate huge
manual effort,
+but also enabling the exploration of much more optimization combinations.
+This automation does not come for free, because we still need to design rules
to generate the search space.
+However, these rules are very general. They are based on static analysis of
the tensor expressions.
+We only need to design a few general rules once and can apply them to almost
all tensor computations in deep learning.</p>
+
+<h2 id="search-process">Search Process</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_overview.png" alt="image" width="40%"
/></p>
+<center> Figure 1. Search Process Overview </center>
+<p></p>
+
+<p>Figure 1. shows the search process of auto-scheduler when optimizing a
whole neural network.
+The system takes deep learning models as input.
+It then partitions the big model into small subgraphs with Relay’s operator
fusion pass.
+A task scheduler is utilized to allocate the time resource for optimizing many
subgraphs.
+At each iteration, it picks a subgraph that has the most potential to increase
the end-to-end performance.
+For this subgraph, we analyze its tensor expression and generate several
sketches for it.
+Then we run evolutionary search with a learned cost model to get a batch of
optimized programs.
+The optimized programs are sent to actual hardware for measurements.
+When the measurements are finished, the profiling results are used as feedback
to update all components of the system.
+This process is repeated iteratively until the optimization converges or we
run out of time budget.
+More technical details can be found in our paper [3] and our code.</p>
+
+<p>It is worth notiing that since the auto-scheduler generates schedules from
scratch,
+it reuses the existing computation definitions in TOPI but not schedule
templates.</p>
+
+<h1 id="benchmark-results">Benchmark Results</h1>
+<p>In this section, we benchmark the performance of AutoTVM and Auto-scheduler.
+The CPU benchmark is done on an AWS c5.9xlarge, which is equipped with an
Intel 18-core skylake 8124-m CPU.
+The GPU benchmark is done on an AWS g4dn.4xlarge, which is equipped with an
NVIDIA T4 GPU.
+All benchmark code, raw data, tuning logs can be found in this repo [2].</p>
+
+<h2 id="performance-of-the-generated-code">Performance of the generated
code</h2>
+<p>We benchmark the fp32 single-batch inference latency on three networks.
+Figure 2 shows the relative speedup of auto-scheduler against AutoTVM.
+We can see auto-scheduler outperforms AutoTVM in all cases with 1.02x to 8.95x
speedup.
+This is because auto-scheduler explores a larger search space, which covers
more efficient combinations
+of optimizations that are missed in TOPI manual templates.
+The BERT-base@GPU is an extreme case where the manual templates are very badly
designed.
+In other words, the manual template for dense layers does not perform well for
the shapes in BERT model.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/code_perf.png" alt="image" width="85%" /></p>
+<center> Figure 2. Code Performance Comparision (Higher is better) </center>
+<p></p>
+
+<h2 id="search-time">Search Time</h2>
+<p>As we know, the search-based approaches can be very time-consuming, so we
also care about the search time.
+It typically takes several hours to let the search converge for a single
neural network.
+Figure 3 compares the search time of AutoTVM and auto-scheduler.
+Auto-scheduler requires much less time to converge in most cases, despite its
larger search space.
+This is mainly because of auto-scheduler has a better cost model and task
scheduler.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_time.png" alt="image" width="85%"
/></p>
+<center> Figure 3. Search Time Comparision (Lower is better) </center>
+<p></p>
+
+<h2 id="more-results">More Results</h2>
+<p>The repo above serves as an internal benchmark tool for TVM, so it only
compares the latest AutoTVM and AutoScheduler.
+You can find results for more libraries and backends in our paper [3].
+Recently, this blog post [4] also tried auto-scheduler on an Apple M1 chip and
got some good results.</p>
+
+<h1 id="conclusion">Conclusion</h1>
+<p>We build TVM auto-scheduler, a system that automatically generates
high-performance code for tensor expressions.
+Compared with the predecessor AutoTVM, auto-scheduler does not require manual
templates.
+Besides, auto-scheduler is capable of generating schedules with better
performance in a shorter time.
+We achieve this by making innovations in the search space construction and
search algorithm.</p>
+
+<p>We are excited about the current performance of auto-scheduler.
+In the future, we are interested in extending the ability of auto-scheduler to
support
+sparse operators, low-precision operators, and dynamic shape better.</p>
+
+<h1 id="links">Links</h1>
+<p>[1] Tutorials: <a
href="https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling";>https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling</a><br
/>
+[2] Benchmark repo: <a
href="https://github.com/tlc-pack/TLCBench";>https://github.com/tlc-pack/TLCBench</a><br
/>
+[3] OSDI Paper: <a href="https://arxiv.org/abs/2006.06762";>Ansor : Generating
High-Performance Tensor Programs for Deep Learning</a><br />
+[4] Results on Apple M1 chip: <a
href="https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d";>https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d</a>.</p>
+
+
+ </div>
+ </div>
+</div>
+</div>
+
+
+
+
+
+
+ <script src="https://code.jquery.com/jquery-2.2.0.min.js";
type="text/javascript"></script>
+ <script
src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js";
integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49"
crossorigin="anonymous"></script>
+ <script
src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js";
integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy"
crossorigin="anonymous"></script>
+ <!-- <script src="./assets/js/slick.js"></script> -->
+ <script src="/assets/js/custome.js"></script>
+ <script async
src="https://www.googletagmanager.com/gtag/js?id=UA-75982049-2";></script>
+ <script>
+ window.dataLayer = window.dataLayer || [];
+ function gtag(){dataLayer.push(arguments);}
+ gtag('js', new Date());
+ gtag('config', 'UA-75982049-2');
+ </script>
+</body>
+<section class="footerSec">
+ <div class="footerHeader">
+ <ul class="container d-flex align-md-items-center justify-content-between
flex-column flex-md-row">
+ <li class="logo">
+
+ <p><a href="/"><img src="/assets/images/logo.svg" alt="logo"
title="logo" /></a></p>
+ </li>
+ <li class="copywrite d-flex align-items-center">
+ <h5 id="apache-software-foundation--all-right-reserved">© 2020 Apache
Software Foundation | All right reserved</h5>
+ </li>
+ </ul>
+
+ </div>
+
+ <ul class="container">
+ <li class="footernote">
+ Copyright © 2020 The Apache Software Foundation. Apache TVM, Apache, the
Apache feather, and the Apache TVM project logo are either trademarks or
registered trademarks of the Apache Software Foundation.</li>
+ </ul>
+
+</section>
+</html>
diff --git a/atom.xml b/atom.xml
index 84cd5f0..cb57f8a 100644
--- a/atom.xml
+++ b/atom.xml
@@ -4,7 +4,7 @@
<title>TVM</title>
<link href="https://tvm.apache.org"; rel="self"/>
<link href="https://tvm.apache.org"/>
- <updated>2021-01-04T16:22:52-05:00</updated>
+ <updated>2021-03-03T01:20:46-08:00</updated>
<id>https://tvm.apache.org</id>
<author>
<name></name>
@@ -13,9 +13,139 @@
<entry>
+ <title>Introducing TVM Auto-scheduler (a.k.a. Ansor)</title>
+ <link href="https://tvm.apache.org/2021/03/03/intro-auto-scheduler"/>
+ <updated>2021-03-03T00:00:00-08:00</updated>
+ <id>https://tvm.apache.org/2021/03/03/intro-auto-scheduler</id>
+ <content type="html"><p>Optimizing the execution speed of deep neural
networks is extremely hard with the growing
+model size, operator diversity, and hardware heterogeneity.
+From a computational perspective, deep neural networks are just layers and
layers of tensor computations.
+These tensor computations, such as matmul and conv2d, can be easily described
by mathematical expressions.
+However, providing high-performance implementations for them on modern
hardware can be very challenging.
+We have to apply various low-level optimizations and utilize special hardware
intrinsics to achieve high performance.
+It takes huge engineering effort to build linear algebra and neural network
acceleration libraries like CuBLAS, CuDNN, oneMKL, and oneDNN.</p>
+
+<p>Our life will be much easier if we can just write mathematical
expressions and have something
+magically turn them into efficient code implementations.
+Three years ago, deep learning compiler TVM and its search module AutoTVM were
built as the first step towards this goal.
+AutoTVM employs a template-based search algorithm to find efficient
implementations for a given tensor computation.
+However, it is a template-based approach, so it still requires domain experts
to implement a non-trivial manual template
+for every operator on every platform.
+Today, there are more than 15k lines of code for these templates in the TVM
code repository.
+Besides being very hard to develop, these templates often have inefficient and
limited search spaces,
+making them unable to achieve optimal performance.</p>
+
+<p>To address the limitations of AutoTVM, we started project Ansor
aiming at a fully automated auto-scheduler for
+generating code for tensor computations.
+Ansor auto-scheduler only takes tensor expressions as input and generates
high-performance code without manual templates.
+We made innovations in the search space construction and search algorithm.
+As a result, the auto-scheduler can achieve better performance with less
search time in a more automated way.</p>
+
+<p>Ansor auto-scheduler is now integrated into Apache TVM as <code
class="language-plaintext
highlighter-rouge">tvm.auto_scheduler</code> package.
+This is a joint effort by collaborators from UC Berkeley, Alibaba, AWS and
OctoML.
+Detailed tutorials are available for Intel CPUs, ARM CPUs, NVIDIA CPUs, and
Mali GPUs on the TVM website [1].
+In this blog post, we will give a high-level introduction and show some
benchmark results.</p>
+
+<h1 id="system-overview">System Overview</h1>
+
+<h2 id="autotvm-vs-auto-scheduler">AutoTVM vs
Auto-scheduler</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/workflow.png" alt="image"
width="75%" /></p>
+<center> Table 1. Workflow Comparision </center>
+<p></p>
+
+<p>Table 1 compares the workflow for generating code for an operator in
AutoTVM and auto-scheduler.
+In AutoTVM, the developer has to go through three steps.
+In step 1, the developer has to write the compute definition in TVM’s tensor
expression language.
+This part is relatively easy because TVM’s tensor expression language looks
just like math expressions.
+In step 2, the developer has to write a schedule template, which typically
consists of 20-100 lines of tricky DSL code.
+This part requires domain expertise of both the target hardware architecture
and operator semantics, so it is difficult.
+The last step, step 3, is automated by a search algorithm.</p>
+
+<p>In auto-scheduler, we eliminate the most difficult step 2 by
automatic search space construction and accelerate step 3 with a better search
algorithm.
+By doing automatic search space construction, we not only eliminate huge
manual effort,
+but also enabling the exploration of much more optimization combinations.
+This automation does not come for free, because we still need to design rules
to generate the search space.
+However, these rules are very general. They are based on static analysis of
the tensor expressions.
+We only need to design a few general rules once and can apply them to almost
all tensor computations in deep learning.</p>
+
+<h2 id="search-process">Search Process</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_overview.png"
alt="image" width="40%" /></p>
+<center> Figure 1. Search Process Overview </center>
+<p></p>
+
+<p>Figure 1. shows the search process of auto-scheduler when optimizing
a whole neural network.
+The system takes deep learning models as input.
+It then partitions the big model into small subgraphs with Relay’s operator
fusion pass.
+A task scheduler is utilized to allocate the time resource for optimizing many
subgraphs.
+At each iteration, it picks a subgraph that has the most potential to increase
the end-to-end performance.
+For this subgraph, we analyze its tensor expression and generate several
sketches for it.
+Then we run evolutionary search with a learned cost model to get a batch of
optimized programs.
+The optimized programs are sent to actual hardware for measurements.
+When the measurements are finished, the profiling results are used as feedback
to update all components of the system.
+This process is repeated iteratively until the optimization converges or we
run out of time budget.
+More technical details can be found in our paper [3] and our code.</p>
+
+<p>It is worth notiing that since the auto-scheduler generates schedules
from scratch,
+it reuses the existing computation definitions in TOPI but not schedule
templates.</p>
+
+<h1 id="benchmark-results">Benchmark Results</h1>
+<p>In this section, we benchmark the performance of AutoTVM and
Auto-scheduler.
+The CPU benchmark is done on an AWS c5.9xlarge, which is equipped with an
Intel 18-core skylake 8124-m CPU.
+The GPU benchmark is done on an AWS g4dn.4xlarge, which is equipped with an
NVIDIA T4 GPU.
+All benchmark code, raw data, tuning logs can be found in this repo
[2].</p>
+
+<h2 id="performance-of-the-generated-code">Performance of the
generated code</h2>
+<p>We benchmark the fp32 single-batch inference latency on three
networks.
+Figure 2 shows the relative speedup of auto-scheduler against AutoTVM.
+We can see auto-scheduler outperforms AutoTVM in all cases with 1.02x to 8.95x
speedup.
+This is because auto-scheduler explores a larger search space, which covers
more efficient combinations
+of optimizations that are missed in TOPI manual templates.
+The BERT-base@GPU is an extreme case where the manual templates are very badly
designed.
+In other words, the manual template for dense layers does not perform well for
the shapes in BERT model.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/code_perf.png"
alt="image" width="85%" /></p>
+<center> Figure 2. Code Performance Comparision (Higher is better)
</center>
+<p></p>
+
+<h2 id="search-time">Search Time</h2>
+<p>As we know, the search-based approaches can be very time-consuming,
so we also care about the search time.
+It typically takes several hours to let the search converge for a single
neural network.
+Figure 3 compares the search time of AutoTVM and auto-scheduler.
+Auto-scheduler requires much less time to converge in most cases, despite its
larger search space.
+This is mainly because of auto-scheduler has a better cost model and task
scheduler.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_time.png"
alt="image" width="85%" /></p>
+<center> Figure 3. Search Time Comparision (Lower is better)
</center>
+<p></p>
+
+<h2 id="more-results">More Results</h2>
+<p>The repo above serves as an internal benchmark tool for TVM, so it
only compares the latest AutoTVM and AutoScheduler.
+You can find results for more libraries and backends in our paper [3].
+Recently, this blog post [4] also tried auto-scheduler on an Apple M1 chip and
got some good results.</p>
+
+<h1 id="conclusion">Conclusion</h1>
+<p>We build TVM auto-scheduler, a system that automatically generates
high-performance code for tensor expressions.
+Compared with the predecessor AutoTVM, auto-scheduler does not require manual
templates.
+Besides, auto-scheduler is capable of generating schedules with better
performance in a shorter time.
+We achieve this by making innovations in the search space construction and
search algorithm.</p>
+
+<p>We are excited about the current performance of auto-scheduler.
+In the future, we are interested in extending the ability of auto-scheduler to
support
+sparse operators, low-precision operators, and dynamic shape better.</p>
+
+<h1 id="links">Links</h1>
+<p>[1] Tutorials: <a
href="https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling">https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling</a><br
/>
+[2] Benchmark repo: <a
href="https://github.com/tlc-pack/TLCBench">https://github.com/tlc-pack/TLCBench</a><br
/>
+[3] OSDI Paper: <a
href="https://arxiv.org/abs/2006.06762">Ansor : Generating
High-Performance Tensor Programs for Deep Learning</a><br />
+[4] Results on Apple M1 chip: <a
href="https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d">https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d</a>.</p>
+
+</content>
+ </entry>
+
+ <entry>
<title>Bring Your Own Datatypes: Enabling Custom Datatype Exploration in
TVM</title>
<link href="https://tvm.apache.org/2020/09/26/bring-your-own-datatypes"/>
- <updated>2020-09-26T00:00:00-04:00</updated>
+ <updated>2020-09-26T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2020/09/26/bring-your-own-datatypes</id>
<content type="html"><p>In this post, we describe the Bring Your Own
Datatypes framework, which enables the use of custom datatypes within
TVM.</p>
@@ -308,7 +438,7 @@ For more documentation about the Bring Your Own Datatypes
framework
<entry>
<title>How to Bring Your Own Codegen to TVM</title>
<link
href="https://tvm.apache.org/2020/07/15/how-to-bring-your-own-codegen-to-tvm"/>
- <updated>2020-07-15T00:00:00-04:00</updated>
+ <updated>2020-07-15T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2020/07/15/how-to-bring-your-own-codegen-to-tvm</id>
<content type="html"><p>To free data scientists from worrying about
the performance when developing a new model, hardware backend providers (e.g.,
Intel, NVIDIA, ARM, etc) either provide kernel libraries such as cuBLAS or
cuDNN with many commonly used deep learning kernels, or provide frameworks such
as DNNL or TensorRT with a graph engine to let users describe their models in a
certain way to achieve high performance. In addition, emerging deep learning
accelerators also have t [...]
@@ -787,7 +917,7 @@ Figure 4: After Graph Partitioning.
<entry>
<title>Bridging PyTorch and TVM</title>
<link href="https://tvm.apache.org/2020/07/14/bert-pytorch-tvm"/>
- <updated>2020-07-14T00:00:00-04:00</updated>
+ <updated>2020-07-14T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2020/07/14/bert-pytorch-tvm</id>
<content type="html">
<p>(A more code-heavy variant is crossposted on the more PyTorch affine
<a
href="https://lernapparat.de/transformers-pytorch-tvm/">Lernapparat</a>,
@@ -1310,7 +1440,7 @@ He is a PyTorch core developer and co-authored <a
href="https://www.mann
<entry>
<title>TinyML - How TVM is Taming Tiny</title>
<link
href="https://tvm.apache.org/2020/06/04/tinyml-how-tvm-is-taming-tiny"/>
- <updated>2020-06-04T00:00:00-04:00</updated>
+ <updated>2020-06-04T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2020/06/04/tinyml-how-tvm-is-taming-tiny</id>
<content type="html">
<p><img src="/images/microtvm/logo.png" alt="microTVM
logo" width="30%" /><br /></p>
@@ -1619,7 +1749,7 @@ Diagram from CMSIS-NN paper showing a 2x2 matrix
multiplication microkernel</
<entry>
<title>Compiling Machine Learning to WASM and WebGPU with Apache TVM</title>
<link
href="https://tvm.apache.org/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu"/>
- <updated>2020-05-14T00:00:00-04:00</updated>
+ <updated>2020-05-14T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu</id>
<content type="html"><p><strong>TLDR</strong></p>
@@ -1706,7 +1836,7 @@ Diagram from CMSIS-NN paper showing a 2x2 matrix
multiplication microkernel</
<entry>
<title>Integrating TVM into PyTorch</title>
<link href="https://tvm.apache.org/2019/05/30/pytorch-frontend"/>
- <updated>2019-05-30T00:00:00-04:00</updated>
+ <updated>2019-05-30T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2019/05/30/pytorch-frontend</id>
<content type="html"><p>As TVM continuously demonstrates improvements
to the efficiency of deep learning execution,
it has become clear that PyTorch stands to benefit from directly leveraging
the compiler stack.
@@ -1808,7 +1938,7 @@ relay_graph = torch_tvm.to_relay(mul, inputs)
<entry>
<title>Automating Optimization of Quantized Deep Learning Models on
CUDA</title>
<link href="https://tvm.apache.org/2019/04/29/opt-cuda-quantized"/>
- <updated>2019-04-29T12:00:00-04:00</updated>
+ <updated>2019-04-29T09:00:00-07:00</updated>
<id>https://tvm.apache.org/2019/04/29/opt-cuda-quantized</id>
<content type="html"><p>Deep learning has been successfully applied
to a variety of tasks.
On real-time scenarios such as inference on autonomous vehicles, the inference
speed of the model is critical.
@@ -1952,7 +2082,7 @@ We show that automatic optimization in TVM makes it easy
and flexible to support
<entry>
<title>TVM Deep Learning Compiler Joins Apache Software Foundation</title>
<link href="https://tvm.apache.org/2019/03/18/tvm-apache-announcement"/>
- <updated>2019-03-18T00:00:00-04:00</updated>
+ <updated>2019-03-18T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2019/03/18/tvm-apache-announcement</id>
<content type="html"><p>There is an increasing need to bring machine
learning to a wide diversity of hardware devices. Current frameworks rely on
vendor-specific operator libraries and optimize for a narrow range of
server-class GPUs. Deploying workloads to new platforms – such as mobile
phones, embedded devices, and accelerators (e.g., FPGAs, ASICs) – requires
significant manual effort.</p>
@@ -1975,7 +2105,7 @@ We show that automatic optimization in TVM makes it easy
and flexible to support
<entry>
<title>TVM Golang Runtime for Deep Learning Deployment</title>
<link href="https://tvm.apache.org/2019/01/19/Golang"/>
- <updated>2019-01-19T00:00:00-05:00</updated>
+ <updated>2019-01-19T00:00:00-08:00</updated>
<id>https://tvm.apache.org/2019/01/19/Golang</id>
<content type="html"><h2
id="introduction">Introduction</h2>
@@ -2145,7 +2275,7 @@ closure as TVM packed function and invoke the same across
programming language b
<entry>
<title>Automating Generation of Low Precision Deep Learning
Operators</title>
<link href="https://tvm.apache.org/2018/12/18/lowprecision-conv"/>
- <updated>2018-12-18T00:00:00-05:00</updated>
+ <updated>2018-12-18T00:00:00-08:00</updated>
<id>https://tvm.apache.org/2018/12/18/lowprecision-conv</id>
<content type="html"><p>As deep learning models grow larger and more
complex, deploying them on low powered phone and IoT
devices becomes challenging because of their limited compute and energy
budgets. A recent trend
@@ -2306,7 +2436,7 @@ Note: x86 doesn’t support a vectorized popcount for this
microarchitecture, so
<entry>
<title>Efficient Privacy-Preserving ML Using TVM</title>
<link href="https://tvm.apache.org/2018/10/09/ml-in-tees"/>
- <updated>2018-10-09T00:00:00-04:00</updated>
+ <updated>2018-10-09T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2018/10/09/ml-in-tees</id>
<content type="html"><p>This post describes Myelin, a framework for
privacy-preserving machine learning in trusted hardware enclaves, and how TVM
makes Myelin fast.
The key idea is that TVM, unlike other popular ML frameworks, compiles models
into lightweight, optimized, and dependency-free libraries which can fit into
resource constrained enclaves.</p>
@@ -2422,7 +2552,7 @@ His research interest is in the general domain of ML on
shared private data, but
<entry>
<title>Automatic Kernel Optimization for Deep Learning on All Hardware
Platforms</title>
<link href="https://tvm.apache.org/2018/10/03/auto-opt-all"/>
- <updated>2018-10-03T00:00:00-04:00</updated>
+ <updated>2018-10-03T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2018/10/03/auto-opt-all</id>
<content type="html"><p>Optimizing the performance of deep neural
network on a diverse range of hardware platforms is still a hard
problem for AI developers. In terms of system support, we are facing a
many-to-many problem here:
@@ -2816,7 +2946,7 @@ for inference deployment. TVM just provides such a
solution.</p>
<entry>
<title>Building a Cross-Framework Deep Learning Compiler via DLPack</title>
<link href="https://tvm.apache.org/2018/08/10/DLPack-Bridge"/>
- <updated>2018-08-10T00:00:00-04:00</updated>
+ <updated>2018-08-10T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2018/08/10/DLPack-Bridge</id>
<content type="html"><p>Deep learning frameworks such as Tensorflow,
PyTorch, and ApacheMxNet provide a
powerful toolbox for quickly prototyping and deploying deep learning models.
@@ -2955,7 +3085,7 @@ support, and can be used to implement convenient
converters, such as
<entry>
<title>VTA: An Open, Customizable Deep Learning Acceleration Stack </title>
<link href="https://tvm.apache.org/2018/07/12/vta-release-announcement"/>
- <updated>2018-07-12T00:00:00-04:00</updated>
+ <updated>2018-07-12T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2018/07/12/vta-release-announcement</id>
<content type="html"><p style="text-align: center">Thierry
Moreau(VTA architect), Tianqi Chen(TVM stack), Ziheng Jiang†(graph
compilation), Luis Vega(cloud deployment)</p>
<p style="text-align: center">Advisors: Luis Ceze, Carlos
Guestrin, Arvind Krishnamurthy</p>
@@ -3097,7 +3227,7 @@ This kind of high-level visibility is essential to system
designers who want to
<entry>
<title>Bringing TVM into TensorFlow for Optimizing Neural Machine
Translation on GPU</title>
<link href="https://tvm.apache.org/2018/03/23/nmt-transformer-optimize"/>
- <updated>2018-03-23T00:00:00-04:00</updated>
+ <updated>2018-03-23T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2018/03/23/nmt-transformer-optimize</id>
<content type="html"><h2 id="author">Author</h2>
@@ -3363,7 +3493,7 @@ C = tvm.compute(
<entry>
<title>Compiling Deep Learning Models to WebGL with TVM</title>
<link href="https://tvm.apache.org/2018/03/12/webgl"/>
- <updated>2018-03-12T00:00:00-04:00</updated>
+ <updated>2018-03-12T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2018/03/12/webgl</id>
<content type="html"><p>Now TVM comes with a brand-new OpenGL/WebGL
backend!
This blog post explains what it is, and what you can achieve with it.</p>
@@ -3479,7 +3609,7 @@ optimizations into the TVM stack.</p>
<entry>
<title>Optimizing Mobile Deep Learning on ARM GPU with TVM</title>
<link href="https://tvm.apache.org/2018/01/16/opt-mali-gpu"/>
- <updated>2018-01-16T00:00:00-05:00</updated>
+ <updated>2018-01-16T00:00:00-08:00</updated>
<id>https://tvm.apache.org/2018/01/16/opt-mali-gpu</id>
<content type="html"><p>With the great success of deep learning, the
demand for
deploying deep neural networks to mobile devices is growing rapidly.
@@ -4053,7 +4183,7 @@ advice and <a
href="https://github.com/yzhliu">Yizhi Liu</a&g
<entry>
<title>Remote Profile and Test Deep Learning Cross Compilation on Mobile
Phones with TVM RPC</title>
<link href="https://tvm.apache.org/2017/11/08/android-rpc-introduction"/>
- <updated>2017-11-08T00:00:00-05:00</updated>
+ <updated>2017-11-08T00:00:00-08:00</updated>
<id>https://tvm.apache.org/2017/11/08/android-rpc-introduction</id>
<content type="html"><p>TVM stack is an end to end compilation stack
to deploy deep learning workloads to all hardware backends.
Thanks to the NNVM compiler support of TVM stack, we can now directly compile
descriptions from deep learning frameworks and compile them to bare metal code.
@@ -4281,7 +4411,7 @@ make jvminstall
<entry>
<title>Bringing AMDGPUs to TVM Stack and NNVM Compiler with ROCm</title>
<link
href="https://tvm.apache.org/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm"/>
- <updated>2017-10-30T00:00:00-04:00</updated>
+ <updated>2017-10-30T00:00:00-07:00</updated>
<id>https://tvm.apache.org/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm</id>
<content type="html"><p style="text-align: center">Aditya
Atluri, Advanced Micro Devices, Inc.</p>
<p style="text-align: center">Masahiro Masuda, Ziosoft,
Inc.</p>
@@ -4504,88 +4634,5 @@ BB0_6:
</content>
</entry>
- <entry>
- <title>NNVM Compiler: Open Compiler for AI Frameworks</title>
- <link href="https://tvm.apache.org/2017/10/06/nnvm-compiler-announcement"/>
- <updated>2017-10-06T11:30:00-04:00</updated>
- <id>https://tvm.apache.org/2017/10/06/nnvm-compiler-announcement</id>
- <content type="html"><p style="text-align: center">Paul G.
Allen School of Computer Science &amp; Engineering, University of
Washington</p>
-<p style="text-align: center">Amazon Web Service AI
team</p>
-<p style="text-align: center">DMLC open-source
community</p>
-
-<p>Deep learning has become ubiquitous and indispensable. We are seeing
a rising need for deploying deep learning workloads on many kinds of platforms
such as mobile phones, GPU, IoT devices and specialized accelerators. Last
month, we announced TVM stack to close the gap between deep learning
frameworks, and the performance- or efficiency-oriented hardware backends. TVM
stack makes it easy to build an end to end compilation for a deep learning
framework. However, we think it wo [...]
-
-<p>Today, UW Allen school and AWS AI team, together with other
contributors, are excited to announce the release of NNVM compiler, an open
deep learning compiler to compile front-end framework workloads directly to
hardware backends. We build it using the two-level intermediate
representation(IR) in the TVM stack.
-The reader is welcome to refer to the <a
href="http://www.tvmlang.org/2017/08/17/tvm-release-announcement.html">original
TVM announcement</a> for more technical details about TVM stack. With
the help of TVM stack, NNVM compiler can:</p>
-
-<ul>
- <li>Represent and optimize the common deep learning workloads in high
level graph IR</li>
- <li>Transform the computation graph to minimize memory utilization,
optimize data layout and fuse computation patterns for different hardware
backends.</li>
- <li>Present an end to end compilation pipeline from front-end deep
learning frameworks to bare metal hardwares.</li>
-</ul>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_compiler_stack.png" alt="image"
width="612px" /></p>
-
-<p>The NNVM compiler can directly take models from deep learning
frameworks such as Apache MXNet.
-It also support model exchange formats such as ONNX and CoreML. ONNX support
enables NNVM to compile deep learning models from PyTorch, Caffe2 and CNTK.
-The CoreML frontend enables deployment of CoreML models to non-iOS
devices.</p>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_compiler_code.png" alt="image"
width="712px" /></p>
-
-<h2 id="separation-of-optimization-and-deployment">Separation
of Optimization and Deployment</h2>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_deploy.png" alt="image"
width="512px" /></p>
-
-<p>NNVM compiler applies graph level and tensor level optimizations and
jointly optimize them to get the best performance. We take a different approach
from existing deep learning frameworks, which packages the graph optimization
with the deployment runtime. NNVM compiler adopts the conventional wisdom from
compiler to separate the optimization from the actual deployment runtime. This
approach offers substantial optimization but still keeps the runtime
lightweight. The compiled mo [...]
-
-<h2 id="performance">Performance</h2>
-
-<p>NNVM compiler is still under active development, and we can expect
more improvements to come, but we have started to see promising results.
-We benchmarked its performance and compared it against Apache MXNet on two
typical hardware configurations: ARM CPU on Raspberry PI and Nvidia GPU on AWS.
Despite the radical architecture difference between these two chips, we can use
the same infrastructure and only need to change the schedule for each type of
hardware.</p>
-
-<h3 id="nvidia-gpu">Nvidia GPU</h3>
-
-<p>GPU benchmarks and schedules are contributed by Leyuan Wang
(AWS/UCDavis) and Yuwei Hu (TuSimple). We compared the NNVM compiler against
Apache MXNet with CUDA8 and cuDNN7 as the backend on Nvidia K80. This is a very
strong baseline, as Apache MXNet turns on auto-tuning to select the best kernel
from CuDNN. We also used the optimized depthwise kernel in MXNet to optimize
MobileNet workload.</p>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_k80_result.png" alt="image"
width="400px" /></p>
-
-<p>As can be seen, NNVM compiler generate code that outperforms Apache
MXNet on K80. These improvements are due to the joint graph level and kernel
level optimizations. It is worth noting that NNVM compiler generates all the
optimized GPU kernels on its own without relying on external libraries like
CuDNN.</p>
-
-<h3 id="raspberry-pi-3b">Raspberry Pi 3b</h3>
-
-<p>The Rasberry Pi compilation stack is contributed by Ziheng
Jiang(AWS/FDU).
-We compared NNVM compiler against Apache MXNet with OpenBLAS and NNPack.
-We explored the setups to get the best performance out of MXNet: we turned on
Winograd convolution in the NNPACK for 3x3 convolutions, enabled
multi-threading and disabled the additional scheduler thread (so all threads
are used by NNPack).</p>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_rasp_result.png" alt="image"
width="400px" /></p>
-
-<p>As can be seen, the code generated by NNVM compiler is two times
faster on ResNet18.
-The gap on MobileNet is mainly due to lack of depthwise convolution in
existing CPU DNN libraries. NNVM compiler takes benefit of direct generating
efficient ARM code directly.</p>
-
-<h2 id="acknowledgement">Acknowledgement</h2>
-<p>This project wouldn’t become possible without our early contributors
in the DMLC community.
-We would like to specially thank Yuwei Hu(TuSimple), Leyuan Wang(AWS/UCDavis),
Joshua Z. Zhang(AWS)
-and Xingjian Shi(HKUST) for their early contributions to the project. We would
also like to thank all the contributors
-to the TVM stack.</p>
-
-<p>We also learnt a lot from the following projects when building NNVM
Compiler.</p>
-<ul>
- <li><a
href="https://github.com/Theano/Theano">Theano</a>: possibly
the earliest compiler for deep learning</li>
- <li><a
href="https://github.com/halide/Halide">Halide</a>: TVM uses
<a href="https://github.com/dmlc/HalideIR">HalideIR</a>
as data structure for
-arithematic simplification and low level lowering. HalideIR is derived from
Halide.
-We also learns from Halide when implementing the lowering pipeline in
TVM.</li>
- <li><a
href="https://github.com/inducer/loopy">Loopy</a>: use of
integer set analysis and its loop transformation primitives.</li>
-</ul>
-
-<h2 id="links">Links</h2>
-<ul>
- <li>Github page of NNVM Compiler: <a
href="https://github.com/dmlc/nnvm">https://github.com/dmlc/nnvm</a></li>
- <li>Github page of TVM: <a
href="https://github.com/dmlc/tvm">https://github.com/dmlc/tvm</a></li>
- <li><a
href="https://news.cs.washington.edu/2017/10/06/allen-school-and-aws-team-up-on-new-nnvm-compiler-for-deep-learning-frameworks/">UW
Allen school blog about NNVM compiler</a></li>
- <li><a
href="https://aws.amazon.com/blogs/ai/introducing-nnvm-compiler-a-new-open-end-to-end-compiler-for-ai-frameworks/">AWS
blogpost about NNVM compiler</a></li>
-</ul>
-</content>
- </entry>
-
</feed>
diff --git a/blog.html b/blog.html
index 8dbab07..ae12173 100644
--- a/blog.html
+++ b/blog.html
@@ -146,6 +146,16 @@
<li>
<span>
+ <a class="post-link" href="/2021/03/03/intro-auto-scheduler">Introducing
TVM Auto-scheduler (a.k.a. Ansor)</a>
+ </span>
+ </br>
+ <span>
+ Mar 3, 2021
+ </span>
+</li>
+
+<li>
+ <span>
<a class="post-link" href="/2020/09/26/bring-your-own-datatypes">Bring
Your Own Datatypes: Enabling Custom Datatype Exploration in TVM</a>
</span>
</br>
diff --git a/community.html b/community.html
index 365bb79..d3e347f 100644
--- a/community.html
+++ b/community.html
@@ -279,6 +279,10 @@ This is a community maintained list of organizations using
and contributing to t
</li>
<li>
+ <img src="/images/community/sjtu.png" />
+ </li>
+
+ <li>
<img src="/images/community/ucberkeley.png" />
</li>
diff --git a/feed.xml b/feed.xml
index a3d90e2..5d387ea 100644
--- a/feed.xml
+++ b/feed.xml
@@ -1,4 +1,124 @@
-<?xml version="1.0" encoding="utf-8"?><feed
xmlns="http://www.w3.org/2005/Atom"; ><generator uri="https://jekyllrb.com/";
version="4.1.1">Jekyll</generator><link href="/feed.xml" rel="self"
type="application/atom+xml" /><link href="/" rel="alternate" type="text/html"
/><updated>2021-01-04T16:22:52-05:00</updated><id>/feed.xml</id><title
type="html">TVM</title><author><name>{"name"=>nil}</name></author><entry><title
type="html">Bring Your Own Datatypes: Enabling Custom Datatype [...]
+<?xml version="1.0" encoding="utf-8"?><feed
xmlns="http://www.w3.org/2005/Atom"; ><generator uri="https://jekyllrb.com/";
version="4.1.1">Jekyll</generator><link href="/feed.xml" rel="self"
type="application/atom+xml" /><link href="/" rel="alternate" type="text/html"
/><updated>2021-03-03T01:20:46-08:00</updated><id>/feed.xml</id><title
type="html">TVM</title><author><name>{"name"=>nil}</name></author><entry><title
type="html">Introducing TVM Auto-scheduler (a.k.a. Ansor)</tit [...]
+model size, operator diversity, and hardware heterogeneity.
+From a computational perspective, deep neural networks are just layers and
layers of tensor computations.
+These tensor computations, such as matmul and conv2d, can be easily described
by mathematical expressions.
+However, providing high-performance implementations for them on modern
hardware can be very challenging.
+We have to apply various low-level optimizations and utilize special hardware
intrinsics to achieve high performance.
+It takes huge engineering effort to build linear algebra and neural network
acceleration libraries like CuBLAS, CuDNN, oneMKL, and oneDNN.</p>
+
+<p>Our life will be much easier if we can just write mathematical
expressions and have something
+magically turn them into efficient code implementations.
+Three years ago, deep learning compiler TVM and its search module AutoTVM were
built as the first step towards this goal.
+AutoTVM employs a template-based search algorithm to find efficient
implementations for a given tensor computation.
+However, it is a template-based approach, so it still requires domain experts
to implement a non-trivial manual template
+for every operator on every platform.
+Today, there are more than 15k lines of code for these templates in the TVM
code repository.
+Besides being very hard to develop, these templates often have inefficient and
limited search spaces,
+making them unable to achieve optimal performance.</p>
+
+<p>To address the limitations of AutoTVM, we started project Ansor
aiming at a fully automated auto-scheduler for
+generating code for tensor computations.
+Ansor auto-scheduler only takes tensor expressions as input and generates
high-performance code without manual templates.
+We made innovations in the search space construction and search algorithm.
+As a result, the auto-scheduler can achieve better performance with less
search time in a more automated way.</p>
+
+<p>Ansor auto-scheduler is now integrated into Apache TVM as <code
class="language-plaintext
highlighter-rouge">tvm.auto_scheduler</code> package.
+This is a joint effort by collaborators from UC Berkeley, Alibaba, AWS and
OctoML.
+Detailed tutorials are available for Intel CPUs, ARM CPUs, NVIDIA CPUs, and
Mali GPUs on the TVM website [1].
+In this blog post, we will give a high-level introduction and show some
benchmark results.</p>
+
+<h1 id="system-overview">System Overview</h1>
+
+<h2 id="autotvm-vs-auto-scheduler">AutoTVM vs
Auto-scheduler</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/workflow.png" alt="image"
width="75%" /></p>
+<center> Table 1. Workflow Comparision </center>
+<p></p>
+
+<p>Table 1 compares the workflow for generating code for an operator in
AutoTVM and auto-scheduler.
+In AutoTVM, the developer has to go through three steps.
+In step 1, the developer has to write the compute definition in TVM’s tensor
expression language.
+This part is relatively easy because TVM’s tensor expression language looks
just like math expressions.
+In step 2, the developer has to write a schedule template, which typically
consists of 20-100 lines of tricky DSL code.
+This part requires domain expertise of both the target hardware architecture
and operator semantics, so it is difficult.
+The last step, step 3, is automated by a search algorithm.</p>
+
+<p>In auto-scheduler, we eliminate the most difficult step 2 by
automatic search space construction and accelerate step 3 with a better search
algorithm.
+By doing automatic search space construction, we not only eliminate huge
manual effort,
+but also enabling the exploration of much more optimization combinations.
+This automation does not come for free, because we still need to design rules
to generate the search space.
+However, these rules are very general. They are based on static analysis of
the tensor expressions.
+We only need to design a few general rules once and can apply them to almost
all tensor computations in deep learning.</p>
+
+<h2 id="search-process">Search Process</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_overview.png"
alt="image" width="40%" /></p>
+<center> Figure 1. Search Process Overview </center>
+<p></p>
+
+<p>Figure 1. shows the search process of auto-scheduler when optimizing
a whole neural network.
+The system takes deep learning models as input.
+It then partitions the big model into small subgraphs with Relay’s operator
fusion pass.
+A task scheduler is utilized to allocate the time resource for optimizing many
subgraphs.
+At each iteration, it picks a subgraph that has the most potential to increase
the end-to-end performance.
+For this subgraph, we analyze its tensor expression and generate several
sketches for it.
+Then we run evolutionary search with a learned cost model to get a batch of
optimized programs.
+The optimized programs are sent to actual hardware for measurements.
+When the measurements are finished, the profiling results are used as feedback
to update all components of the system.
+This process is repeated iteratively until the optimization converges or we
run out of time budget.
+More technical details can be found in our paper [3] and our code.</p>
+
+<p>It is worth notiing that since the auto-scheduler generates schedules
from scratch,
+it reuses the existing computation definitions in TOPI but not schedule
templates.</p>
+
+<h1 id="benchmark-results">Benchmark Results</h1>
+<p>In this section, we benchmark the performance of AutoTVM and
Auto-scheduler.
+The CPU benchmark is done on an AWS c5.9xlarge, which is equipped with an
Intel 18-core skylake 8124-m CPU.
+The GPU benchmark is done on an AWS g4dn.4xlarge, which is equipped with an
NVIDIA T4 GPU.
+All benchmark code, raw data, tuning logs can be found in this repo
[2].</p>
+
+<h2 id="performance-of-the-generated-code">Performance of the
generated code</h2>
+<p>We benchmark the fp32 single-batch inference latency on three
networks.
+Figure 2 shows the relative speedup of auto-scheduler against AutoTVM.
+We can see auto-scheduler outperforms AutoTVM in all cases with 1.02x to 8.95x
speedup.
+This is because auto-scheduler explores a larger search space, which covers
more efficient combinations
+of optimizations that are missed in TOPI manual templates.
+The BERT-base@GPU is an extreme case where the manual templates are very badly
designed.
+In other words, the manual template for dense layers does not perform well for
the shapes in BERT model.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/code_perf.png"
alt="image" width="85%" /></p>
+<center> Figure 2. Code Performance Comparision (Higher is better)
</center>
+<p></p>
+
+<h2 id="search-time">Search Time</h2>
+<p>As we know, the search-based approaches can be very time-consuming,
so we also care about the search time.
+It typically takes several hours to let the search converge for a single
neural network.
+Figure 3 compares the search time of AutoTVM and auto-scheduler.
+Auto-scheduler requires much less time to converge in most cases, despite its
larger search space.
+This is mainly because of auto-scheduler has a better cost model and task
scheduler.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_time.png"
alt="image" width="85%" /></p>
+<center> Figure 3. Search Time Comparision (Lower is better)
</center>
+<p></p>
+
+<h2 id="more-results">More Results</h2>
+<p>The repo above serves as an internal benchmark tool for TVM, so it
only compares the latest AutoTVM and AutoScheduler.
+You can find results for more libraries and backends in our paper [3].
+Recently, this blog post [4] also tried auto-scheduler on an Apple M1 chip and
got some good results.</p>
+
+<h1 id="conclusion">Conclusion</h1>
+<p>We build TVM auto-scheduler, a system that automatically generates
high-performance code for tensor expressions.
+Compared with the predecessor AutoTVM, auto-scheduler does not require manual
templates.
+Besides, auto-scheduler is capable of generating schedules with better
performance in a shorter time.
+We achieve this by making innovations in the search space construction and
search algorithm.</p>
+
+<p>We are excited about the current performance of auto-scheduler.
+In the future, we are interested in extending the ability of auto-scheduler to
support
+sparse operators, low-precision operators, and dynamic shape better.</p>
+
+<h1 id="links">Links</h1>
+<p>[1] Tutorials: <a
href="https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling">https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling</a><br
/>
+[2] Benchmark repo: <a
href="https://github.com/tlc-pack/TLCBench">https://github.com/tlc-pack/TLCBench</a><br
/>
+[3] OSDI Paper: <a
href="https://arxiv.org/abs/2006.06762">Ansor : Generating
High-Performance Tensor Programs for Deep Learning</a><br />
+[4] Results on Apple M1 chip: <a
href="https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d">https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d</a>.</p></content><author><name>Lianmin
Zheng, Chengfan Jia, Minmin Sun, Zhao Wu, Cody Hao Yu</name></author><summary
type="html">Optimizing the execution speed of deep neural networks i [...]
<h2 id="introduction">Introduction</h2>
@@ -282,7 +402,7 @@ For more documentation about the Bring Your Own Datatypes
framework
<p><a
href="https://posithub.org/docs/BeatingFloatingPoint.pdf"
target="_blank">Beating Floating Point at its Own Game: Posit
Arithmetic</a> <a href="#fnref:posit"
class="reversefootnote"
role="doc-backlink">&#8617;</a></p>
</li>
</ol>
-</div></content><author><name>Gus Smith, Andrew
Liu</name></author><summary type="html">In this post, we describe the Bring
Your Own Datatypes framework, which enables the use of custom datatypes within
TVM.</summary></entry><entry><title type="html">How to Bring Your Own Codegen
to TVM</title><link href="/2020/07/15/how-to-bring-your-own-codegen-to-tvm"
rel="alternate" type="text/html" title="How to Bring Your Own Codegen to TVM"
/><published>2020-07-15T00:00:00-04:00</published>< [...]
+</div></content><author><name>Gus Smith, Andrew
Liu</name></author><summary type="html">In this post, we describe the Bring
Your Own Datatypes framework, which enables the use of custom datatypes within
TVM.</summary></entry><entry><title type="html">How to Bring Your Own Codegen
to TVM</title><link href="/2020/07/15/how-to-bring-your-own-codegen-to-tvm"
rel="alternate" type="text/html" title="How to Bring Your Own Codegen to TVM"
/><published>2020-07-15T00:00:00-07:00</published>< [...]
<p>However, users have to learn a new programming interface when they
attempt to work on a new kernel library or a device. As a result, the demand
for a unified programming interface becomes more and more important to let all
users and hardware backend providers stand on the same page.</p>
@@ -751,7 +871,7 @@ Figure 4: After Graph Partitioning.
<h2 id="acknowledgment">Acknowledgment</h2>
-<p>We would like to thank our colleague Animesh Jain for valuable
discussions in the framework design; Tianqi Chen and Jared Roesch from OctoML
for system design discussions and prototyping; Masahiro Masuda from the TVM
community to help code review and improve the DNNL integration. We would also
like to thank Ramana Radhakrishnan, Matthew Barrett, Manupa Karunaratne, and
Luke Hutton from ARM, U.K. for contributing several helpful ideas, related
Relay passes, and the Arm Compute Li [...]
+<p>We would like to thank our colleague Animesh Jain for valuable
discussions in the framework design; Tianqi Chen and Jared Roesch from OctoML
for system design discussions and prototyping; Masahiro Masuda from the TVM
community to help code review and improve the DNNL integration. We would also
like to thank Ramana Radhakrishnan, Matthew Barrett, Manupa Karunaratne, and
Luke Hutton from ARM, U.K. for contributing several helpful ideas, related
Relay passes, and the Arm Compute Li [...]
the Jupyter Notebook to follow along is on <a
href="https://github.com/t-vi/pytorch-tvmisc/tree/master/transformers-pytorch-tvm/">github</a>.)</p>
<p>Some of the most intriguing applications of Artificial Intelligence
have been in Natural Language Processing.
@@ -1264,7 +1384,7 @@ one would want to re-do cheap computation, most
prominently point-wise computati
<h1 id="author">Author</h1>
<p><a href="https://lernapparat.de/">Thomas
Viehmann</a> is the founder of <a
href="https://mathinf.eu/">MathInf GmbH</a>, Munich,
Germany, a boutique training and consultancy firm focusing on Machine Learning
and PyTorch.
-He is a PyTorch core developer and co-authored <a
href="https://www.manning.com/books/deep-learning-with-pytorch">Deep
Learning with PyTorch</a>, which currently available as <a
href="https://pytorch.org/deep-learning-with-pytorch">free
download from the PyTorch
website</a>.</p></content><author><name>Thomas Viehmann, MathInf
GmbH</name></author><summary type="html"></summary></entry><entry><title
type="html">TinyML - How TVM is Taming Ti [...]
+He is a PyTorch core developer and co-authored <a
href="https://www.manning.com/books/deep-learning-with-pytorch">Deep
Learning with PyTorch</a>, which currently available as <a
href="https://pytorch.org/deep-learning-with-pytorch">free
download from the PyTorch
website</a>.</p></content><author><name>Thomas Viehmann, MathInf
GmbH</name></author><summary type="html"></summary></entry><entry><title
type="html">TinyML - How TVM is Taming Ti [...]
<p>The proliferation of low-cost, AI-powered consumer devices has led to
widespread interest in “bare-metal” (low-power, often without an operating
system) devices among ML researchers and practitioners. While it is already
possible for experts to run <em>some</em> models on
<em>some</em> bare-metal devices, optimizing models for diverse
sets of devices is challenging, often requiring manually optimized
device-specific libraries. And for those platforms wi [...]
@@ -1563,7 +1683,7 @@ Diagram from CMSIS-NN paper showing a 2x2 matrix
multiplication microkernel</
<li><a
href="https://homes.cs.washington.edu/~moreau/">Thierry
Moreau</a>, for mentoring me during my time at OctoML.</li>
<li><a
href="https://homes.cs.washington.edu/~vegaluis/">Luis
Vega</a>, for teaching me the fundamentals of interacting with
microcontrollers.</li>
<li><a
href="https://www.linkedin.com/in/themadrasi/?originalSubdomain=uk">Ramana
Radhakrishnan</a>, for supplying the Arm hardware used in our
experiments and for providing guidance on its usage.</li>
-</ul></content><author><name>Logan Weber and Andrew Reusch,
OctoML</name></author><summary type="html"></summary></entry><entry><title
type="html">Compiling Machine Learning to WASM and WebGPU with Apache
TVM</title><link
href="/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu"
rel="alternate" type="text/html" title="Compiling Machine Learning to WASM and
WebGPU with Apache TVM"
/><published>2020-05-14T00:00:00-04:00</published><updated>2020-05-14T00:00:00-04:00</upd
[...]
+</ul></content><author><name>Logan Weber and Andrew Reusch,
OctoML</name></author><summary type="html"></summary></entry><entry><title
type="html">Compiling Machine Learning to WASM and WebGPU with Apache
TVM</title><link
href="/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu"
rel="alternate" type="text/html" title="Compiling Machine Learning to WASM and
WebGPU with Apache TVM"
/><published>2020-05-14T00:00:00-07:00</published><updated>2020-05-14T00:00:00-07:00</upd
[...]
<p>We introduced support for WASM and WebGPU to the Apache TVM deep
learning compiler. Our experiments shows that TVM’s WebGPU backend can get
<strong>close to native</strong> <strong>GPU
performance</strong> when deploying models to the web.</p>
@@ -1641,7 +1761,7 @@ Diagram from CMSIS-NN paper showing a 2x2 matrix
multiplication microkernel</
<h2 id="acknowledgement">Acknowledgement</h2>
-<p>We would like to thank the emscripten project for providing the WASM
compilation infrastructures as well as the JS library support on the web. We
would also like to thank the WebGPU community for various helpful discussions.
Thanks to Fletcher Haynes for valuable feedbacks to the
post.</p></content><author><name>Tianqi Chen and Jared Roesch,
OctoML</name></author><summary type="html">TLDR</summary></entry><entry><title
type="html">Integrating TVM into PyTorch</title><link [...]
+<p>We would like to thank the emscripten project for providing the WASM
compilation infrastructures as well as the JS library support on the web. We
would also like to thank the WebGPU community for various helpful discussions.
Thanks to Fletcher Haynes for valuable feedbacks to the
post.</p></content><author><name>Tianqi Chen and Jared Roesch,
OctoML</name></author><summary type="html">TLDR</summary></entry><entry><title
type="html">Integrating TVM into PyTorch</title><link [...]
it has become clear that PyTorch stands to benefit from directly leveraging
the compiler stack.
A major tenet of PyTorch is providing seamless and robust integrations that
don’t get in the user’s way.
To that end, PyTorch now has an official TVM-based backend, <a
href="https://github.com/pytorch/tvm">torch_tvm</a>.</p>
@@ -1733,7 +1853,7 @@ def mul(a, b, c):
# via script
relay_graph = torch_tvm.to_relay(mul, inputs)
-</code></pre></div></div></content><author><name>Bram
Wasti</name></author><summary type="html">As TVM continuously demonstrates
improvements to the efficiency of deep learning execution, it has become clear
that PyTorch stands to benefit from directly leveraging the compiler stack. A
major tenet of PyTorch is providing seamless and robust integrations that don’t
get in the user’s way. To that end, PyTorch now has an official TVM-based
backend, torch_tvm.</summary [...]
+</code></pre></div></div></content><author><name>Bram
Wasti</name></author><summary type="html">As TVM continuously demonstrates
improvements to the efficiency of deep learning execution, it has become clear
that PyTorch stands to benefit from directly leveraging the compiler stack. A
major tenet of PyTorch is providing seamless and robust integrations that don’t
get in the user’s way. To that end, PyTorch now has an official TVM-based
backend, torch_tvm.</summary [...]
On real-time scenarios such as inference on autonomous vehicles, the inference
speed of the model is critical.
Network quantization is an effective approach to accelerating deep learning
models.
In quantized models, both data and model parameters are represented with low
precision data types such as <code class="language-plaintext
highlighter-rouge">int8</code> and <code
class="language-plaintext highlighter-rouge">float16</code>.
@@ -1868,7 +1988,7 @@ We show that automatic optimization in TVM makes it easy
and flexible to support
</ul>
<h1 id="bio--acknowledgement">Bio &amp;
Acknowledgement</h1>
-<p><a href="https://wuwei.io/">Wuwei Lin</a> is an
undergraduate student at SJTU. He is currently an intern at TuSimple. The
author has many thanks to <a
href="https://homes.cs.washington.edu/~tqchen/">Tianqi
Chen</a> and <a
href="https://homes.cs.washington.edu/~eqy/">Eddie Yan</a>
for their reviews.</p></content><author><name>Wuwei
Lin</name></author><summary type="html">Deep learning has been successfully ap
[...]
+<p><a href="https://wuwei.io/">Wuwei Lin</a> is an
undergraduate student at SJTU. He is currently an intern at TuSimple. The
author has many thanks to <a
href="https://homes.cs.washington.edu/~tqchen/">Tianqi
Chen</a> and <a
href="https://homes.cs.washington.edu/~eqy/">Eddie Yan</a>
for their reviews.</p></content><author><name>Wuwei
Lin</name></author><summary type="html">Deep learning has been successfully ap
[...]
<p>TVM is an open source deep learning compiler stack that closes the
gap between the productivity-focused deep learning frameworks, and the
performance- or efficiency-oriented hardware backends. Today, we are glad to
announce that the TVM community has decided to move on to Apache incubator, and
becomes an Apache(incubating) project.</p>
@@ -1882,7 +2002,7 @@ We show that automatic optimization in TVM makes it easy
and flexible to support
<p>We would like to take this chance to thank the Allen School for
supporting the SAMPL team that gave birth to the TVM project. We would also
like to thank the Halide project which provided the basis for TVM’s loop-level
IR and initial code generation. We would like to thank our Apache incubator
mentors for introducing the project to Apache and providing useful guidance.
Finally, we would like to thank the TVM community and all of the organizations,
as listed above, that supported [...]
-<p>See also the <a
href="https://news.cs.washington.edu/2019/03/18/allen-schools-tvm-deep-learning-compiler-framework-transitions-to-apache/">Allen
School news about the transition here</a>, <a
href="https://sampl.cs.washington.edu/tvmconf/#about-tvmconf">TVM
conference program slides and recordings</a>, and <a
href="https://tvm.apache.org/docs//contribute/community.html">our
community guideline here</a>. Follow us o [...]
+<p>See also the <a
href="https://news.cs.washington.edu/2019/03/18/allen-schools-tvm-deep-learning-compiler-framework-transitions-to-apache/">Allen
School news about the transition here</a>, <a
href="https://sampl.cs.washington.edu/tvmconf/#about-tvmconf">TVM
conference program slides and recordings</a>, and <a
href="https://tvm.apache.org/docs//contribute/community.html">our
community guideline here</a>. Follow us o [...]
<p>TVM is an open deep learning compiler stack to compile various deep
learning models from different
frameworks to CPU, GPU or specialized accelerators. TVM supports model
compilation from a wide range
@@ -2043,155 +2163,4 @@ closure as TVM packed function and invoke the same
across programming language b
<li>[5] <a
href="https://blog.learngoprogramming.com/golang-variadic-funcs-how-to-patterns-369408f19085">Go
Variadic Functions</a></li>
<li>[6] <a
href="https://github.com/jdeng/gomxnet">CFFI
Ref</a></li>
<li>[7] <a
href="https://golang.org/pkg/runtime/#SetFinalizer">Go
Finalizers</a></li>
-</ul></content><author><name>Siva</name></author><summary
type="html">Introduction</summary></entry><entry><title type="html">Automating
Generation of Low Precision Deep Learning Operators</title><link
href="/2018/12/18/lowprecision-conv" rel="alternate" type="text/html"
title="Automating Generation of Low Precision Deep Learning Operators"
/><published>2018-12-18T00:00:00-05:00</published><updated>2018-12-18T00:00:00-05:00</updated><id>/2018/12/18/lowprecision-conv</id><content
ty [...]
-devices becomes challenging because of their limited compute and energy
budgets. A recent trend
- in deep learning is the use of extremely quantized models that
operate on inputs and
- weights of a few bits, with networks like XNOR-Net, DoReFa-Net, and
HWGQ-Net making steady
-progress improving accuracy.</p>
-
-<p>An example of a low precision graph snippet is below. The low
precision convolution takes in
-quantized data and bitpacks into the proper data layout for an efficient
bitserial convolution.
-The output is in a higher precision and traditional deep learning layers such
as batch normalization and ReLu are applied to it, before being re-quantized
and sent through another low precision operator.</p>
-
-<p style="text-align: center"><img
src="/images/low-precision/workflow.png" alt="image"
width="50%" /></p>
-<center> Low precision convolution pipeline.</center>
-<p></p>
-
-<p>Theoretically, low precision operators use less operations than
-floating point operators, leading many to believe they can achieve up
tremendous speedups.
-However, deep learning frameworks leverage decades of engineering work
through low level
-BLAS and LAPACK libraries that are incredibly well optimized, and CPUs
include intrinsic
-instructions to accelerate these tasks. In practice, it is not simple
to develop low-level
-operators such as convolutions that are competitive with 8-bit quantized
or even floating
-point operators.
-In this post we introduce our approach to automatically generating
optimized
-low precision convolutions for CPUs. We declare our low precision operators
so that they compute
-on efficiently stored low precision inputs, and describe a schedule that
describes a search space
-of implementation parameters. We rely on AutoTVM to quickly search the space
and find optimized
-parameters for the particular convolution, precision, and backend.</p>
-
-<h2 id="bitserial-computation-background">Bitserial
Computation Background</h2>
-
-<p>The core of low precision models is the bitserial dot product
that enables convolution and
-dense operators to be computed using only bitwise operations and popcount.
- Typically, a dot product is computed by element wise multiplication of two
vectors followed by
- summing all the elements, like the simple example below. If all the data is
binary, the input
- vectors can be packed into single integer, and the dot product can be
computed by bitwise-anding
- the packed inputs and counting the number of 1’s in the result using popcount.
-Note: Depending how the input data is quantized, bitwise-xnor may be used
instead of bitwise-and.</p>
-
-<p style="text-align: center"><img
src="/images/low-precision/binary-dotproduct.png"
alt="image" width="50%" /></p>
-<center> Binary dot product.</center>
-<p></p>
-
-<p>Arbitrary precision dot products can be computed in this fashion by
first separating input data
-into bitplanes. Once in this representation we can compute dotproduct by
summing weighted binary
-dot products between the bitplanes of A and B. The number of binary
dotproducts grows with the
-product of A and B’s precision, so this method is only practical for very low
precision data.</p>
-
-<p style="text-align: center"><img
src="/images/low-precision/bitserial-dotproduct.png"
alt="image" width="50%" /></p>
-<center> Bitserial dot product.</center>
-<p></p>
-
-<h2 id="defining-operators-in-tvm">Defining Operators in
TVM</h2>
-<p>Before the computation, input data needs to be bitpacked so that the
bitplanes of the input data
-can be accessed and are packed into a supported datatype such as a uint8 or
uint32. We provide
-a flexible bitpacking operator that takes arbitrary size input tensors and
returns a bitpacked
-tensor where the user specifies which axis the bitplanes should be.</p>
-
-<p style="text-align: center"><img
src="/images/low-precision/bitpack.png" alt="image"
width="50%" /></p>
-<center> Different bitpacked layouts.</center>
-<p></p>
-
-<p>Once in this bitpacked format the low precision convolution can be
computed bitserially.
-For this demo, that data is packed along the input channel and the bitplanes
are added to the
-innermost axis, and the data is packed into 32-bit integers. The bitserial
convolution is computed
-similar to a normal convolution, but the bitwise-and (&amp;) replaces
multiplication, and we use
-popcount to accumulate values in the packed data. The bitplane axes become
additional reduction axes
-and compute the binary dot products between different bitplanes of the input
and kernel.
-Finally, the output is computed in an unpacked format and in higher
precision.</p>
-
-<div class="language-python highlighter-rouge"><div
class="highlight"><pre
class="highlight"><code><span
class="n">Input_bitpacked</span> <span
class="o">=</span> <span
class="n">bitpack</span><span
class="p">(</span><span
class="n">Input</span><span
class="p">,</span> <span class="n">acti [...]
-<span class="n">Weights_bitpacked</span> <span
class="o">=</span> <span
class="n">bitpack</span><span
class="p">(</span><span
class="n">Filter</span><span
class="p">,</span> <span
class="n">weight_bits</span><span
class="p">,</span> <span
class="n">pack_axis</span><span class="o"& [...]
-<span class="n">batch</span><span
class="p">,</span> <span
class="n">in_height</span><span
class="p">,</span> <span
class="n">in_width</span><span
class="p">,</span> <span
class="n">in_channel_q</span><span
class="p">,</span> <span
class="n">_</span> <span
class="o">=</span& [...]
-<span class="n">kernel_h</span><span
class="p">,</span> <span
class="n">kernel_w</span><span
class="p">,</span> <span
class="n">_</span><span
class="p">,</span> <span
class="n">num_filter</span><span
class="p">,</span> <span
class="n">_</span> <span
class="o">=</span> < [...]
-
-<span class="n">stride_h</span><span
class="p">,</span> <span
class="n">stride_w</span> <span
class="o">=</span> <span
class="n">stride</span>
-<span class="n">pad_top</span><span
class="p">,</span> <span
class="n">pad_left</span><span
class="p">,</span> <span
class="n">pad_down</span><span
class="p">,</span> <span
class="n">pad_right</span> <span
class="o">=</span> <span
class="n">get_pad_tuple</span><span
class="p">( [...]
-
-<span class="c1"># Computing the output shape
-</span><span class="n">out_channel</span> <span
class="o">=</span> <span
class="n">num_filter</span>
-<span class="n">out_height</span> <span
class="o">=</span> <span
class="n">simplify</span><span
class="p">((</span><span
class="n">in_height</span> <span
class="o">-</span> <span
class="n">kernel_h</span> <span
class="o">+</span> <span
class="n">pad_top</span> <span class="o">+
[...]
-<span class="n">out_width</span> <span
class="o">=</span> <span
class="n">simplify</span><span
class="p">((</span><span
class="n">in_width</span> <span
class="o">-</span> <span
class="n">kernel_w</span> <span
class="o">+</span> <span
class="n">pad_left</span> <span class="o">+&
[...]
-<span class="n">pad_before</span> <span
class="o">=</span> <span
class="p">[</span><span
class="mi">0</span><span
class="p">,</span> <span
class="n">pad_top</span><span
class="p">,</span> <span
class="n">pad_left</span><span
class="p">,</span> <span
class="mi">0</span>< [...]
-<span class="n">pad_after</span> <span
class="o">=</span> <span
class="p">[</span><span
class="mi">0</span><span
class="p">,</span> <span
class="n">pad_down</span><span
class="p">,</span> <span
class="n">pad_right</span><span
class="p">,</span> <span
class="mi">0</span>&l [...]
-<span class="n">Input_padded</span> <span
class="o">=</span> <span
class="n">pad</span><span
class="p">(</span><span
class="n">Input_bitpacked</span><span
class="p">,</span> <span
class="n">pad_before</span><span
class="p">,</span> <span
class="n">pad_after</span><span class="p"&g
[...]
-
-<span class="c1"># Treat the bitplane axes like additional
reduction axes
-</span><span class="n">rc</span> <span
class="o">=</span> <span
class="n">tvm</span><span
class="p">.</span><span
class="n">reduce_axis</span><span
class="p">((</span><span
class="mi">0</span><span
class="p">,</span> <span
class="n">in_channel_q</span><span
class="p">),&l [...]
-<span class="n">ry</span> <span
class="o">=</span> <span
class="n">tvm</span><span
class="p">.</span><span
class="n">reduce_axis</span><span
class="p">((</span><span
class="mi">0</span><span
class="p">,</span> <span
class="n">kernel_h</span><span
class="p">),</span> <s [...]
-<span class="n">rx</span> <span
class="o">=</span> <span
class="n">tvm</span><span
class="p">.</span><span
class="n">reduce_axis</span><span
class="p">((</span><span
class="mi">0</span><span
class="p">,</span> <span
class="n">kernel_w</span><span
class="p">),</span> <s [...]
-<span class="n">ib</span> <span
class="o">=</span> <span
class="n">tvm</span><span
class="p">.</span><span
class="n">reduce_axis</span><span
class="p">((</span><span
class="mi">0</span><span
class="p">,</span> <span
class="n">input_bits</span><span
class="p">),</span> < [...]
-<span class="n">wb</span> <span
class="o">=</span> <span
class="n">tvm</span><span
class="p">.</span><span
class="n">reduce_axis</span><span
class="p">((</span><span
class="mi">0</span><span
class="p">,</span> <span
class="n">weight_bits</span><span
class="p">),</span> &l [...]
-
-
-<span class="n">tvm</span><span
class="p">.</span><span
class="n">compute</span><span
class="p">((</span><span
class="n">batch</span><span
class="p">,</span> <span
class="n">out_height</span><span
class="p">,</span> <span
class="n">out_width</span><span
class="p">,</span> [...]
- <span class="n">tvm</span><span
class="p">.</span><span
class="nb">sum</span><span
class="p">(</span><span
class="n">tvm</span><span
class="p">.</span><span
class="n">popcount</span><span
class="p">(</span>
- <span
class="n">Input_padded</span><span
class="p">[</span><span
class="n">nn</span><span
class="p">,</span> <span
class="n">yy</span> <span
class="o">*</span> <span
class="n">stride_h</span> <span
class="o">+</span> <span
class="n">ry</span><span class="p">,<
[...]
- <span
class="n">Weights_bitpacked</span><span
class="p">[</span><span
class="n">ry</span><span
class="p">,</span> <span
class="n">rx</span><span
class="p">,</span> <span
class="n">rc</span><span
class="p">,</span> <span
class="n">ff</span><span class="p">,</sp
[...]
- <span class="n">axis</span><span
class="o">=</span><span
class="p">[</span><span
class="n">rc</span><span
class="p">,</span> <span
class="n">ry</span><span
class="p">,</span> <span
class="n">rx</span><span
class="p">,</span> <span
class="n">wb</span><spa [...]
-
-</code></pre></div></div>
-
-<p>In our schedule we apply common optimizations like vectorization and
memory tiling to provide better
-memory locality and take advantage of SIMD units. Some of these optimizations
such as tiling,
-require parameters that need to be tuned to for the specific
microarchitecture. We expose these
-parameters as knobs to TVM and use AutoTVM to automatically tune all the
parameters simultaneously.</p>
-
-<p>Finally, we can craft small microkernels to replace the innermost
loop(s) of computation and schedule
- them using TVM’s tensorize primitive. Since, compilers often produce
suboptimal code, people can
- often write short assembly sequences that are more efficient. These
microkernels often take advantage
- of new intrinsics that are being introduced to help accelerate deep learning
workloads and use
- them clever ways to improve memory accesses or reduce the number instructions
required.</p>
-
-<h2 id="results">Results</h2>
-
-<h3 id="raspberry-pi">Raspberry Pi</h3>
-<p>Convolution speedups on Raspberry Pi 3B compared to 16-bit integer
TVM implementation.
-Workload are convolution layers from ResNet18.</p>
-
-<p style="text-align: center"><img
src="/images/low-precision/rasp-conv.png" alt="image"
width="50%" /></p>
-<center> Speedup of low precision convolutions on a Raspberry Pi
compared to 16-bit TVM implementation.</center>
-<p></p>
-
-<p>2-bit activation, 1-bit weight convolution speedups on Raspberry Pi
3B compared to hand optimized implementation from <a
href="https://arxiv.org/pdf/1712.02427.pdf">High performance
ultra-low-precision convolutions
-on mobile devices.</a>.
-Workload are convolution layers from ResNet18.</p>
-
-<p style="text-align: center"><img
src="/images/low-precision/rasp-conv-2.png" alt="image"
width="50%" /></p>
-<center> Speedup of 2-bit weight 1-bit activation Raspberry Pi
convolutions against a hand optimized implementation.</center>
-<p></p>
-
-<h3 id="x86">x86</h3>
-
-<p>Convolution speedups on x86 compared to a 32-bit floating point TVM
implementation.
-Note: x86 doesn’t support a vectorized popcount for this microarchitecture, so
speedups are lower.</p>
-<p style="text-align: center"><img
src="/images/low-precision/x86-conv.png" alt="image"
width="50%" /></p>
-<center> Speedup of x86 low precision convolutions compared to a 32-bit
floating point TVM implementation.</center>
-<p></p>
-
-<h2 id="show-me-the-code">Show me the code</h2>
-
-<ul>
- <li><a
href="https://github.com/apache/incubator-tvm/blob/main/topi/python/topi/nn/bitserial_conv2d.py">TOPI
bitserial convolution</a></li>
- <li><a
href="https://github.com/apache/incubator-tvm/blob/main/topi/python/topi/arm_cpu/bitserial_conv2d.py">TOPI
ARM cpu bitserial convolution</a></li>
-</ul>
-
-<h2 id="references">References</h2>
-
-<ul>
- <li>[1] <a
href="https://arxiv.org/abs/1810.11066">Automating Generation of
Low Precision Deep Learning Operators</a></li>
- <li>[2] <a
href="https://arxiv.org/abs/1603.05279">XNOR-Net</a></li>
- <li>[3] <a
href="https://arxiv.org/abs/1702.00953">HWGQ</a></li>
- <li>[4] <a
href="https://arxiv.org/abs/1606.06160">DoReFa</a></li>
-</ul></content><author><name>Meghan Cowan</name></author><summary
type="html">As deep learning models grow larger and more complex, deploying
them on low powered phone and IoT devices becomes challenging because of their
limited compute and energy budgets. A recent trend in deep learning is the use
of extremely quantized models that operate on inputs and weights of a few bits,
with networks like XNOR-Net, DoReFa-Net, and HWGQ-Net making steady progress
improving accuracy.</summary> [...]
\ No newline at end of file
+</ul></content><author><name>Siva</name></author><summary
type="html">Introduction</summary></entry></feed>
\ No newline at end of file
diff --git a/images/community/sjtu.png b/images/community/sjtu.png
new file mode 100644
index 0000000..0de00de
Binary files /dev/null and b/images/community/sjtu.png differ
diff --git a/images/intro-auto-scheduler/code_perf.png
b/images/intro-auto-scheduler/code_perf.png
new file mode 100644
index 0000000..d070a6e
Binary files /dev/null and b/images/intro-auto-scheduler/code_perf.png differ
diff --git a/images/intro-auto-scheduler/search_overview.png
b/images/intro-auto-scheduler/search_overview.png
new file mode 100644
index 0000000..7b6f56d
Binary files /dev/null and b/images/intro-auto-scheduler/search_overview.png
differ
diff --git a/images/intro-auto-scheduler/search_time.png
b/images/intro-auto-scheduler/search_time.png
new file mode 100644
index 0000000..4bd700b
Binary files /dev/null and b/images/intro-auto-scheduler/search_time.png differ
diff --git a/images/intro-auto-scheduler/workflow.png
b/images/intro-auto-scheduler/workflow.png
new file mode 100644
index 0000000..b2c7b26
Binary files /dev/null and b/images/intro-auto-scheduler/workflow.png differ
diff --git a/rss.xml b/rss.xml
index f2dfac7..2173b21 100644
--- a/rss.xml
+++ b/rss.xml
@@ -5,12 +5,142 @@
<description>TVM - </description>
<link>https://tvm.apache.org</link>
<atom:link href="https://tvm.apache.org"; rel="self"
type="application/rss+xml" />
- <lastBuildDate>Mon, 04 Jan 2021 16:22:52 -0500</lastBuildDate>
- <pubDate>Mon, 04 Jan 2021 16:22:52 -0500</pubDate>
+ <lastBuildDate>Wed, 03 Mar 2021 01:20:46 -0800</lastBuildDate>
+ <pubDate>Wed, 03 Mar 2021 01:20:46 -0800</pubDate>
<ttl>60</ttl>
<item>
+ <title>Introducing TVM Auto-scheduler (a.k.a. Ansor)</title>
+ <description><p>Optimizing the execution speed of deep
neural networks is extremely hard with the growing
+model size, operator diversity, and hardware heterogeneity.
+From a computational perspective, deep neural networks are just layers and
layers of tensor computations.
+These tensor computations, such as matmul and conv2d, can be easily described
by mathematical expressions.
+However, providing high-performance implementations for them on modern
hardware can be very challenging.
+We have to apply various low-level optimizations and utilize special hardware
intrinsics to achieve high performance.
+It takes huge engineering effort to build linear algebra and neural network
acceleration libraries like CuBLAS, CuDNN, oneMKL, and oneDNN.</p>
+
+<p>Our life will be much easier if we can just write mathematical
expressions and have something
+magically turn them into efficient code implementations.
+Three years ago, deep learning compiler TVM and its search module AutoTVM were
built as the first step towards this goal.
+AutoTVM employs a template-based search algorithm to find efficient
implementations for a given tensor computation.
+However, it is a template-based approach, so it still requires domain experts
to implement a non-trivial manual template
+for every operator on every platform.
+Today, there are more than 15k lines of code for these templates in the TVM
code repository.
+Besides being very hard to develop, these templates often have inefficient and
limited search spaces,
+making them unable to achieve optimal performance.</p>
+
+<p>To address the limitations of AutoTVM, we started project Ansor
aiming at a fully automated auto-scheduler for
+generating code for tensor computations.
+Ansor auto-scheduler only takes tensor expressions as input and generates
high-performance code without manual templates.
+We made innovations in the search space construction and search algorithm.
+As a result, the auto-scheduler can achieve better performance with less
search time in a more automated way.</p>
+
+<p>Ansor auto-scheduler is now integrated into Apache TVM as <code
class="language-plaintext
highlighter-rouge">tvm.auto_scheduler</code> package.
+This is a joint effort by collaborators from UC Berkeley, Alibaba, AWS and
OctoML.
+Detailed tutorials are available for Intel CPUs, ARM CPUs, NVIDIA CPUs, and
Mali GPUs on the TVM website [1].
+In this blog post, we will give a high-level introduction and show some
benchmark results.</p>
+
+<h1 id="system-overview">System Overview</h1>
+
+<h2 id="autotvm-vs-auto-scheduler">AutoTVM vs
Auto-scheduler</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/workflow.png" alt="image"
width="75%" /></p>
+<center> Table 1. Workflow Comparision </center>
+<p></p>
+
+<p>Table 1 compares the workflow for generating code for an operator in
AutoTVM and auto-scheduler.
+In AutoTVM, the developer has to go through three steps.
+In step 1, the developer has to write the compute definition in TVM’s tensor
expression language.
+This part is relatively easy because TVM’s tensor expression language looks
just like math expressions.
+In step 2, the developer has to write a schedule template, which typically
consists of 20-100 lines of tricky DSL code.
+This part requires domain expertise of both the target hardware architecture
and operator semantics, so it is difficult.
+The last step, step 3, is automated by a search algorithm.</p>
+
+<p>In auto-scheduler, we eliminate the most difficult step 2 by
automatic search space construction and accelerate step 3 with a better search
algorithm.
+By doing automatic search space construction, we not only eliminate huge
manual effort,
+but also enabling the exploration of much more optimization combinations.
+This automation does not come for free, because we still need to design rules
to generate the search space.
+However, these rules are very general. They are based on static analysis of
the tensor expressions.
+We only need to design a few general rules once and can apply them to almost
all tensor computations in deep learning.</p>
+
+<h2 id="search-process">Search Process</h2>
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_overview.png"
alt="image" width="40%" /></p>
+<center> Figure 1. Search Process Overview </center>
+<p></p>
+
+<p>Figure 1. shows the search process of auto-scheduler when optimizing
a whole neural network.
+The system takes deep learning models as input.
+It then partitions the big model into small subgraphs with Relay’s operator
fusion pass.
+A task scheduler is utilized to allocate the time resource for optimizing many
subgraphs.
+At each iteration, it picks a subgraph that has the most potential to increase
the end-to-end performance.
+For this subgraph, we analyze its tensor expression and generate several
sketches for it.
+Then we run evolutionary search with a learned cost model to get a batch of
optimized programs.
+The optimized programs are sent to actual hardware for measurements.
+When the measurements are finished, the profiling results are used as feedback
to update all components of the system.
+This process is repeated iteratively until the optimization converges or we
run out of time budget.
+More technical details can be found in our paper [3] and our code.</p>
+
+<p>It is worth notiing that since the auto-scheduler generates schedules
from scratch,
+it reuses the existing computation definitions in TOPI but not schedule
templates.</p>
+
+<h1 id="benchmark-results">Benchmark Results</h1>
+<p>In this section, we benchmark the performance of AutoTVM and
Auto-scheduler.
+The CPU benchmark is done on an AWS c5.9xlarge, which is equipped with an
Intel 18-core skylake 8124-m CPU.
+The GPU benchmark is done on an AWS g4dn.4xlarge, which is equipped with an
NVIDIA T4 GPU.
+All benchmark code, raw data, tuning logs can be found in this repo
[2].</p>
+
+<h2 id="performance-of-the-generated-code">Performance of the
generated code</h2>
+<p>We benchmark the fp32 single-batch inference latency on three
networks.
+Figure 2 shows the relative speedup of auto-scheduler against AutoTVM.
+We can see auto-scheduler outperforms AutoTVM in all cases with 1.02x to 8.95x
speedup.
+This is because auto-scheduler explores a larger search space, which covers
more efficient combinations
+of optimizations that are missed in TOPI manual templates.
+The BERT-base@GPU is an extreme case where the manual templates are very badly
designed.
+In other words, the manual template for dense layers does not perform well for
the shapes in BERT model.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/code_perf.png"
alt="image" width="85%" /></p>
+<center> Figure 2. Code Performance Comparision (Higher is better)
</center>
+<p></p>
+
+<h2 id="search-time">Search Time</h2>
+<p>As we know, the search-based approaches can be very time-consuming,
so we also care about the search time.
+It typically takes several hours to let the search converge for a single
neural network.
+Figure 3 compares the search time of AutoTVM and auto-scheduler.
+Auto-scheduler requires much less time to converge in most cases, despite its
larger search space.
+This is mainly because of auto-scheduler has a better cost model and task
scheduler.</p>
+
+<p style="text-align: center"><img
src="/images/intro-auto-scheduler/search_time.png"
alt="image" width="85%" /></p>
+<center> Figure 3. Search Time Comparision (Lower is better)
</center>
+<p></p>
+
+<h2 id="more-results">More Results</h2>
+<p>The repo above serves as an internal benchmark tool for TVM, so it
only compares the latest AutoTVM and AutoScheduler.
+You can find results for more libraries and backends in our paper [3].
+Recently, this blog post [4] also tried auto-scheduler on an Apple M1 chip and
got some good results.</p>
+
+<h1 id="conclusion">Conclusion</h1>
+<p>We build TVM auto-scheduler, a system that automatically generates
high-performance code for tensor expressions.
+Compared with the predecessor AutoTVM, auto-scheduler does not require manual
templates.
+Besides, auto-scheduler is capable of generating schedules with better
performance in a shorter time.
+We achieve this by making innovations in the search space construction and
search algorithm.</p>
+
+<p>We are excited about the current performance of auto-scheduler.
+In the future, we are interested in extending the ability of auto-scheduler to
support
+sparse operators, low-precision operators, and dynamic shape better.</p>
+
+<h1 id="links">Links</h1>
+<p>[1] Tutorials: <a
href="https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling">https://tvm.apache.org/docs/tutorials/index.html#autoscheduler-template-free-auto-scheduling</a><br
/>
+[2] Benchmark repo: <a
href="https://github.com/tlc-pack/TLCBench">https://github.com/tlc-pack/TLCBench</a><br
/>
+[3] OSDI Paper: <a
href="https://arxiv.org/abs/2006.06762">Ansor : Generating
High-Performance Tensor Programs for Deep Learning</a><br />
+[4] Results on Apple M1 chip: <a
href="https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d">https://medium.com/octoml/on-the-apple-m1-beating-apples-core-ml-4-with-30-model-performance-improvements-9d94af7d1b2d</a>.</p>
+
+</description>
+
<link>https://tvm.apache.org/2021/03/03/intro-auto-scheduler</link>
+
<guid>https://tvm.apache.org/2021/03/03/intro-auto-scheduler</guid>
+ <pubDate>Wed, 03 Mar 2021 00:00:00 -0800</pubDate>
+ </item>
+
+ <item>
<title>Bring Your Own Datatypes: Enabling Custom Datatype
Exploration in TVM</title>
<description><p>In this post, we describe the Bring Your
Own Datatypes framework, which enables the use of custom datatypes within
TVM.</p>
@@ -300,7 +430,7 @@ For more documentation about the Bring Your Own Datatypes
framework
</description>
<link>https://tvm.apache.org/2020/09/26/bring-your-own-datatypes</link>
<guid>https://tvm.apache.org/2020/09/26/bring-your-own-datatypes</guid>
- <pubDate>Sat, 26 Sep 2020 00:00:00 -0400</pubDate>
+ <pubDate>Sat, 26 Sep 2020 00:00:00 -0700</pubDate>
</item>
<item>
@@ -779,7 +909,7 @@ Figure 4: After Graph Partitioning.
</description>
<link>https://tvm.apache.org/2020/07/15/how-to-bring-your-own-codegen-to-tvm</link>
<guid>https://tvm.apache.org/2020/07/15/how-to-bring-your-own-codegen-to-tvm</guid>
- <pubDate>Wed, 15 Jul 2020 00:00:00 -0400</pubDate>
+ <pubDate>Wed, 15 Jul 2020 00:00:00 -0700</pubDate>
</item>
<item>
@@ -1302,7 +1432,7 @@ He is a PyTorch core developer and co-authored <a
href="https://www.mann
</description>
<link>https://tvm.apache.org/2020/07/14/bert-pytorch-tvm</link>
<guid>https://tvm.apache.org/2020/07/14/bert-pytorch-tvm</guid>
- <pubDate>Tue, 14 Jul 2020 00:00:00 -0400</pubDate>
+ <pubDate>Tue, 14 Jul 2020 00:00:00 -0700</pubDate>
</item>
<item>
@@ -1611,7 +1741,7 @@ Diagram from CMSIS-NN paper showing a 2x2 matrix
multiplication microkernel</
</description>
<link>https://tvm.apache.org/2020/06/04/tinyml-how-tvm-is-taming-tiny</link>
<guid>https://tvm.apache.org/2020/06/04/tinyml-how-tvm-is-taming-tiny</guid>
- <pubDate>Thu, 04 Jun 2020 00:00:00 -0400</pubDate>
+ <pubDate>Thu, 04 Jun 2020 00:00:00 -0700</pubDate>
</item>
<item>
@@ -1698,7 +1828,7 @@ Diagram from CMSIS-NN paper showing a 2x2 matrix
multiplication microkernel</
</description>
<link>https://tvm.apache.org/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu</link>
<guid>https://tvm.apache.org/2020/05/14/compiling-machine-learning-to-webassembly-and-webgpu</guid>
- <pubDate>Thu, 14 May 2020 00:00:00 -0400</pubDate>
+ <pubDate>Thu, 14 May 2020 00:00:00 -0700</pubDate>
</item>
<item>
@@ -1800,7 +1930,7 @@ relay_graph = torch_tvm.to_relay(mul, inputs)
</description>
<link>https://tvm.apache.org/2019/05/30/pytorch-frontend</link>
<guid>https://tvm.apache.org/2019/05/30/pytorch-frontend</guid>
- <pubDate>Thu, 30 May 2019 00:00:00 -0400</pubDate>
+ <pubDate>Thu, 30 May 2019 00:00:00 -0700</pubDate>
</item>
<item>
@@ -1944,7 +2074,7 @@ We show that automatic optimization in TVM makes it easy
and flexible to support
</description>
<link>https://tvm.apache.org/2019/04/29/opt-cuda-quantized</link>
<guid>https://tvm.apache.org/2019/04/29/opt-cuda-quantized</guid>
- <pubDate>Mon, 29 Apr 2019 12:00:00 -0400</pubDate>
+ <pubDate>Mon, 29 Apr 2019 09:00:00 -0700</pubDate>
</item>
<item>
@@ -1967,7 +2097,7 @@ We show that automatic optimization in TVM makes it easy
and flexible to support
</description>
<link>https://tvm.apache.org/2019/03/18/tvm-apache-announcement</link>
<guid>https://tvm.apache.org/2019/03/18/tvm-apache-announcement</guid>
- <pubDate>Mon, 18 Mar 2019 00:00:00 -0400</pubDate>
+ <pubDate>Mon, 18 Mar 2019 00:00:00 -0700</pubDate>
</item>
<item>
@@ -2137,7 +2267,7 @@ closure as TVM packed function and invoke the same across
programming language b
</description>
<link>https://tvm.apache.org/2019/01/19/Golang</link>
<guid>https://tvm.apache.org/2019/01/19/Golang</guid>
- <pubDate>Sat, 19 Jan 2019 00:00:00 -0500</pubDate>
+ <pubDate>Sat, 19 Jan 2019 00:00:00 -0800</pubDate>
</item>
<item>
@@ -2298,7 +2428,7 @@ Note: x86 doesn’t support a vectorized popcount for this
microarchitecture, so
</description>
<link>https://tvm.apache.org/2018/12/18/lowprecision-conv</link>
<guid>https://tvm.apache.org/2018/12/18/lowprecision-conv</guid>
- <pubDate>Tue, 18 Dec 2018 00:00:00 -0500</pubDate>
+ <pubDate>Tue, 18 Dec 2018 00:00:00 -0800</pubDate>
</item>
<item>
@@ -2414,7 +2544,7 @@ His research interest is in the general domain of ML on
shared private data, but
</description>
<link>https://tvm.apache.org/2018/10/09/ml-in-tees</link>
<guid>https://tvm.apache.org/2018/10/09/ml-in-tees</guid>
- <pubDate>Tue, 09 Oct 2018 00:00:00 -0400</pubDate>
+ <pubDate>Tue, 09 Oct 2018 00:00:00 -0700</pubDate>
</item>
<item>
@@ -2808,7 +2938,7 @@ for inference deployment. TVM just provides such a
solution.</p>
</description>
<link>https://tvm.apache.org/2018/10/03/auto-opt-all</link>
<guid>https://tvm.apache.org/2018/10/03/auto-opt-all</guid>
- <pubDate>Wed, 03 Oct 2018 00:00:00 -0400</pubDate>
+ <pubDate>Wed, 03 Oct 2018 00:00:00 -0700</pubDate>
</item>
<item>
@@ -2947,7 +3077,7 @@ support, and can be used to implement convenient
converters, such as
</description>
<link>https://tvm.apache.org/2018/08/10/DLPack-Bridge</link>
<guid>https://tvm.apache.org/2018/08/10/DLPack-Bridge</guid>
- <pubDate>Fri, 10 Aug 2018 00:00:00 -0400</pubDate>
+ <pubDate>Fri, 10 Aug 2018 00:00:00 -0700</pubDate>
</item>
<item>
@@ -3089,7 +3219,7 @@ This kind of high-level visibility is essential to system
designers who want to
</description>
<link>https://tvm.apache.org/2018/07/12/vta-release-announcement</link>
<guid>https://tvm.apache.org/2018/07/12/vta-release-announcement</guid>
- <pubDate>Thu, 12 Jul 2018 00:00:00 -0400</pubDate>
+ <pubDate>Thu, 12 Jul 2018 00:00:00 -0700</pubDate>
</item>
<item>
@@ -3355,7 +3485,7 @@ C = tvm.compute(
</description>
<link>https://tvm.apache.org/2018/03/23/nmt-transformer-optimize</link>
<guid>https://tvm.apache.org/2018/03/23/nmt-transformer-optimize</guid>
- <pubDate>Fri, 23 Mar 2018 00:00:00 -0400</pubDate>
+ <pubDate>Fri, 23 Mar 2018 00:00:00 -0700</pubDate>
</item>
<item>
@@ -3471,7 +3601,7 @@ optimizations into the TVM stack.</p>
</description>
<link>https://tvm.apache.org/2018/03/12/webgl</link>
<guid>https://tvm.apache.org/2018/03/12/webgl</guid>
- <pubDate>Mon, 12 Mar 2018 00:00:00 -0400</pubDate>
+ <pubDate>Mon, 12 Mar 2018 00:00:00 -0700</pubDate>
</item>
<item>
@@ -4045,7 +4175,7 @@ advice and <a
href="https://github.com/yzhliu">Yizhi Liu</a&g
</description>
<link>https://tvm.apache.org/2018/01/16/opt-mali-gpu</link>
<guid>https://tvm.apache.org/2018/01/16/opt-mali-gpu</guid>
- <pubDate>Tue, 16 Jan 2018 00:00:00 -0500</pubDate>
+ <pubDate>Tue, 16 Jan 2018 00:00:00 -0800</pubDate>
</item>
<item>
@@ -4273,7 +4403,7 @@ make jvminstall
</description>
<link>https://tvm.apache.org/2017/11/08/android-rpc-introduction</link>
<guid>https://tvm.apache.org/2017/11/08/android-rpc-introduction</guid>
- <pubDate>Wed, 08 Nov 2017 00:00:00 -0500</pubDate>
+ <pubDate>Wed, 08 Nov 2017 00:00:00 -0800</pubDate>
</item>
<item>
@@ -4499,90 +4629,7 @@ BB0_6:
</description>
<link>https://tvm.apache.org/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm</link>
<guid>https://tvm.apache.org/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-and-NNVM-Compiler-with-ROCm</guid>
- <pubDate>Mon, 30 Oct 2017 00:00:00 -0400</pubDate>
- </item>
-
- <item>
- <title>NNVM Compiler: Open Compiler for AI Frameworks</title>
- <description><p style="text-align:
center">Paul G. Allen School of Computer Science &amp; Engineering,
University of Washington</p>
-<p style="text-align: center">Amazon Web Service AI
team</p>
-<p style="text-align: center">DMLC open-source
community</p>
-
-<p>Deep learning has become ubiquitous and indispensable. We are seeing
a rising need for deploying deep learning workloads on many kinds of platforms
such as mobile phones, GPU, IoT devices and specialized accelerators. Last
month, we announced TVM stack to close the gap between deep learning
frameworks, and the performance- or efficiency-oriented hardware backends. TVM
stack makes it easy to build an end to end compilation for a deep learning
framework. However, we think it wo [...]
-
-<p>Today, UW Allen school and AWS AI team, together with other
contributors, are excited to announce the release of NNVM compiler, an open
deep learning compiler to compile front-end framework workloads directly to
hardware backends. We build it using the two-level intermediate
representation(IR) in the TVM stack.
-The reader is welcome to refer to the <a
href="http://www.tvmlang.org/2017/08/17/tvm-release-announcement.html">original
TVM announcement</a> for more technical details about TVM stack. With
the help of TVM stack, NNVM compiler can:</p>
-
-<ul>
- <li>Represent and optimize the common deep learning workloads in high
level graph IR</li>
- <li>Transform the computation graph to minimize memory utilization,
optimize data layout and fuse computation patterns for different hardware
backends.</li>
- <li>Present an end to end compilation pipeline from front-end deep
learning frameworks to bare metal hardwares.</li>
-</ul>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_compiler_stack.png" alt="image"
width="612px" /></p>
-
-<p>The NNVM compiler can directly take models from deep learning
frameworks such as Apache MXNet.
-It also support model exchange formats such as ONNX and CoreML. ONNX support
enables NNVM to compile deep learning models from PyTorch, Caffe2 and CNTK.
-The CoreML frontend enables deployment of CoreML models to non-iOS
devices.</p>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_compiler_code.png" alt="image"
width="712px" /></p>
-
-<h2 id="separation-of-optimization-and-deployment">Separation
of Optimization and Deployment</h2>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_deploy.png" alt="image"
width="512px" /></p>
-
-<p>NNVM compiler applies graph level and tensor level optimizations and
jointly optimize them to get the best performance. We take a different approach
from existing deep learning frameworks, which packages the graph optimization
with the deployment runtime. NNVM compiler adopts the conventional wisdom from
compiler to separate the optimization from the actual deployment runtime. This
approach offers substantial optimization but still keeps the runtime
lightweight. The compiled mo [...]
-
-<h2 id="performance">Performance</h2>
-
-<p>NNVM compiler is still under active development, and we can expect
more improvements to come, but we have started to see promising results.
-We benchmarked its performance and compared it against Apache MXNet on two
typical hardware configurations: ARM CPU on Raspberry PI and Nvidia GPU on AWS.
Despite the radical architecture difference between these two chips, we can use
the same infrastructure and only need to change the schedule for each type of
hardware.</p>
-
-<h3 id="nvidia-gpu">Nvidia GPU</h3>
-
-<p>GPU benchmarks and schedules are contributed by Leyuan Wang
(AWS/UCDavis) and Yuwei Hu (TuSimple). We compared the NNVM compiler against
Apache MXNet with CUDA8 and cuDNN7 as the backend on Nvidia K80. This is a very
strong baseline, as Apache MXNet turns on auto-tuning to select the best kernel
from CuDNN. We also used the optimized depthwise kernel in MXNet to optimize
MobileNet workload.</p>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_k80_result.png" alt="image"
width="400px" /></p>
-
-<p>As can be seen, NNVM compiler generate code that outperforms Apache
MXNet on K80. These improvements are due to the joint graph level and kernel
level optimizations. It is worth noting that NNVM compiler generates all the
optimized GPU kernels on its own without relying on external libraries like
CuDNN.</p>
-
-<h3 id="raspberry-pi-3b">Raspberry Pi 3b</h3>
-
-<p>The Rasberry Pi compilation stack is contributed by Ziheng
Jiang(AWS/FDU).
-We compared NNVM compiler against Apache MXNet with OpenBLAS and NNPack.
-We explored the setups to get the best performance out of MXNet: we turned on
Winograd convolution in the NNPACK for 3x3 convolutions, enabled
multi-threading and disabled the additional scheduler thread (so all threads
are used by NNPack).</p>
-
-<p style="text-align: center"><img
src="/images/nnvm/nnvm_rasp_result.png" alt="image"
width="400px" /></p>
-
-<p>As can be seen, the code generated by NNVM compiler is two times
faster on ResNet18.
-The gap on MobileNet is mainly due to lack of depthwise convolution in
existing CPU DNN libraries. NNVM compiler takes benefit of direct generating
efficient ARM code directly.</p>
-
-<h2 id="acknowledgement">Acknowledgement</h2>
-<p>This project wouldn’t become possible without our early contributors
in the DMLC community.
-We would like to specially thank Yuwei Hu(TuSimple), Leyuan Wang(AWS/UCDavis),
Joshua Z. Zhang(AWS)
-and Xingjian Shi(HKUST) for their early contributions to the project. We would
also like to thank all the contributors
-to the TVM stack.</p>
-
-<p>We also learnt a lot from the following projects when building NNVM
Compiler.</p>
-<ul>
- <li><a
href="https://github.com/Theano/Theano">Theano</a>: possibly
the earliest compiler for deep learning</li>
- <li><a
href="https://github.com/halide/Halide">Halide</a>: TVM uses
<a href="https://github.com/dmlc/HalideIR">HalideIR</a>
as data structure for
-arithematic simplification and low level lowering. HalideIR is derived from
Halide.
-We also learns from Halide when implementing the lowering pipeline in
TVM.</li>
- <li><a
href="https://github.com/inducer/loopy">Loopy</a>: use of
integer set analysis and its loop transformation primitives.</li>
-</ul>
-
-<h2 id="links">Links</h2>
-<ul>
- <li>Github page of NNVM Compiler: <a
href="https://github.com/dmlc/nnvm">https://github.com/dmlc/nnvm</a></li>
- <li>Github page of TVM: <a
href="https://github.com/dmlc/tvm">https://github.com/dmlc/tvm</a></li>
- <li><a
href="https://news.cs.washington.edu/2017/10/06/allen-school-and-aws-team-up-on-new-nnvm-compiler-for-deep-learning-frameworks/">UW
Allen school blog about NNVM compiler</a></li>
- <li><a
href="https://aws.amazon.com/blogs/ai/introducing-nnvm-compiler-a-new-open-end-to-end-compiler-for-ai-frameworks/">AWS
blogpost about NNVM compiler</a></li>
-</ul>
-</description>
-
<link>https://tvm.apache.org/2017/10/06/nnvm-compiler-announcement</link>
-
<guid>https://tvm.apache.org/2017/10/06/nnvm-compiler-announcement</guid>
- <pubDate>Fri, 06 Oct 2017 11:30:00 -0400</pubDate>
+ <pubDate>Mon, 30 Oct 2017 00:00:00 -0700</pubDate>
</item>
diff --git a/sitemap.txt b/sitemap.txt
index bfad106..db8795d 100644
--- a/sitemap.txt
+++ b/sitemap.txt
@@ -16,6 +16,7 @@ https://tvm.apache.org/vta
https://tvm.apache.org/feed.xml
https://tvm.apache.org/css/custom.css.map
+https://tvm.apache.org/2021/03/03/intro-auto-scheduler
https://tvm.apache.org/2020/09/26/bring-your-own-datatypes
https://tvm.apache.org/2020/07/15/how-to-bring-your-own-codegen-to-tvm
https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
