[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-28 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416998175



##
File path: python/tvm/exec/rpc_server.py
##
@@ -100,27 +101,34 @@ def server_shutdown():
 parser.add_argument('--port-end', type=int, default=9199,
 help='The end search port of the RPC')
 parser.add_argument('--tracker', type=str,
-help="The address of RPC tracker in host:port format. "
- "e.g. (10.77.1.234:9190)")
+help=('The address of RPC tracker in host:port format. 
'
+  'e.g. (10.77.1.234:9190)'))
 parser.add_argument('--key', type=str, default="",
-help="The key used to identify the device type in 
tracker.")
+help='The key used to identify the device type in 
tracker.')
 parser.add_argument('--silent', action='store_true',
-help="Whether run in silent mode.")
+help='Whether run in silent mode.')
 parser.add_argument('--load-library', type=str,
-help="Additional library to load")
+help='Additional library to load')

Review comment:
   done

##
File path: python/tvm/micro/device/arm/stm32f746xx.py
##
@@ -36,23 +55,41 @@ def create_micro_lib(obj_path, src_path, lib_type, 
options=None):
 
 options : Optional[List[str]]
 additional options to pass to GCC
+
+lib_src_paths : Optional[List[str]]
+TODO
 """
 if options is None:
 options = []
+else:
+options = list(options)
+
 options += [
-"-mcpu=cortex-m7",
-"-mlittle-endian",
-"-mfloat-abi=hard",
-"-mfpu=fpv5-sp-d16",
-"-mthumb",
-"-gdwarf-5",
+# TODO(weberlo): make a debug flag
+'-O2',
+'-march=armv7e-m',

Review comment:
   removed





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-28 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416996008



##
File path: python/tvm/exec/rpc_server.py
##
@@ -100,27 +101,34 @@ def server_shutdown():
 parser.add_argument('--port-end', type=int, default=9199,
 help='The end search port of the RPC')
 parser.add_argument('--tracker', type=str,
-help="The address of RPC tracker in host:port format. "
- "e.g. (10.77.1.234:9190)")
+help=('The address of RPC tracker in host:port format. 
'
+  'e.g. (10.77.1.234:9190)'))
 parser.add_argument('--key', type=str, default="",
-help="The key used to identify the device type in 
tracker.")
+help='The key used to identify the device type in 
tracker.')
 parser.add_argument('--silent', action='store_true',
-help="Whether run in silent mode.")
+help='Whether run in silent mode.')
 parser.add_argument('--load-library', type=str,
-help="Additional library to load")
+help='Additional library to load')

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-28 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416995111



##
File path: python/tvm/autotvm/measure/local_executor.py
##
@@ -145,6 +145,7 @@ def submit(self, func, *args, **kwargs):
 if not self.do_fork:
 return LocalFutureNoFork(func(*args, **kwargs))
 
+# TODO why they choose a queue size of 2? add a comment

Review comment:
   I added a comment to this effect





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416140904



##
File path: src/runtime/micro/micro_session.cc
##
@@ -209,59 +246,124 @@ MicroSession::~MicroSession() {
   low_level_device_ = nullptr;
 }
 
-double MicroSession::PushToExecQueue(DevPtr func_ptr, const TVMArgs& args) {
+void MicroSession::PushToTaskQueue(TargetPtr func_ptr, const TVMArgs& args) {
   if (thumb_mode_) {
+// TODO(areusch): should be |=
 func_ptr += 1;
   }
+  TargetVal func_dev_addr = func_ptr.value();
+
+  std::tuple arg_field_addrs = 
EncoderAppend(_args_encoder_, args);
+  TargetVal arg_values_dev_addr{std::get<0>(arg_field_addrs).value()};
+  TargetVal arg_type_codes_dev_addr{std::get<1>(arg_field_addrs).value()};
+
+  task_queue_.push_back(
+  DevTask {
+.func = func_dev_addr,
+.arg_values = arg_values_dev_addr,
+.arg_type_codes = arg_type_codes_dev_addr,
+.num_args = args.num_args
+  });
+
+  if (task_queue_.size() == MicroSession::kTaskQueueCapacity) {
+FlushTaskQueue();
+  }
+}
 
-  // Create an allocator stream for the memory region after the most recent
-  // allocation in the args section.
-  DevPtr args_addr = GetAllocator(SectionKind::kArgs)->curr_end_addr();
-  TargetDataLayoutEncoder encoder(args_addr, word_size_);
-
-  std::tuple arg_field_addrs = EncoderAppend(, args);
-
-  // Flush `stream` to device memory.
-  DevPtr stream_dev_addr =
-  GetAllocator(SectionKind::kArgs)->Allocate(encoder.buf_size());
-  low_level_device()->Write(stream_dev_addr,
-reinterpret_cast(encoder.data()),
-encoder.buf_size());
-
-  TargetVal arg_values_dev_addr = std::get<0>(arg_field_addrs).value();
-  TargetVal arg_type_codes_dev_addr = std::get<1>(arg_field_addrs).value();
-  if (word_size_ == 4) {
-UTVMTask32 task = {
-  .func = func_ptr.value().val32,
-  .arg_values = arg_values_dev_addr.val32,
-  .arg_type_codes = arg_type_codes_dev_addr.val32,
-  .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
-  } else if (word_size_ == 8) {
-UTVMTask64 task = {
-  .func = func_ptr.value().val64,
-  .arg_values = arg_values_dev_addr.val64,
-  .arg_type_codes = arg_type_codes_dev_addr.val64,
-  .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
+void MicroSession::FlushTaskQueue() {
+  if (task_queue_.size() == 0) {
+// nothing to run
+return;
+  }
+  if (word_size_.bytes() == 4) {
+FlushTaskQueuePriv();
+  } else if (word_size_.bytes() == 8) {
+FlushTaskQueuePriv();
   }
+}
 
-  DevPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
-  DevPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
+template 
+void MicroSession::FlushTaskQueuePriv() {
+  // std::cout << "[MicroSession::FlushTaskQueue]" << std::endl;
+  std::vector prepped_tasks;
+  for (const auto& task : task_queue_) {
+prepped_tasks.push_back(T(task));
+  }
+
+  // Flush `args` to device memory.
+  low_level_device()->Write(
+  batch_args_encoder_.start_addr(),
+  reinterpret_cast(batch_args_encoder_.data()),
+  batch_args_encoder_.buf_size());
+
+  // Flush `tasks` to device memory.
+//  runtime_symbol_map_.Dump(std::cout);

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416140211



##
File path: python/tvm/autotvm/tuner/callback.py
##
@@ -144,12 +144,12 @@ def __del__(self):
 def _callback(tuner, inputs, results):
 ctx.ct += len(inputs)
 
-flops = 0
+flops = float("inf")

Review comment:
   reverted, this was cruft that snuck in

##
File path: python/tvm/autotvm/tuner/callback.py
##
@@ -144,12 +144,12 @@ def __del__(self):
 def _callback(tuner, inputs, results):
 ctx.ct += len(inputs)
 
-flops = 0
+flops = float("inf")
 for inp, res in zip(inputs, results):
 if res.error_no == 0:
-flops = inp.task.flop / np.mean(res.costs)
+flops = min(inp.task.flop / np.mean(res.costs), flops)

Review comment:
   reverted





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416140782



##
File path: python/tvm/autotvm/tuner/tuner.py
##
@@ -150,7 +150,15 @@ def tune(self, n_trial, measure_option, 
early_stopping=None, callbacks=(), si_pr
  i + k + 1, si_prefix, format_si_prefix(flops, 
si_prefix),
  format_si_prefix(self.best_flops, si_prefix), 
res, config)
 
-i += len(results)
+num_successes = 0
+for result in results:
+if isinstance(result.costs[0], float):
+num_successes += 1
+if num_successes != len(results):
+logger.debug('not counting %d failures towards trial count',
+ len(results) - num_successes)
+i += num_successes
+

Review comment:
   reverted





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416139282



##
File path: topi/python/topi/arm_cpu/cortex_m7/micro_kernel/gemm.py
##
@@ -0,0 +1,221 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, no-value-for-parameter
+"""Defines gemm intrinsics for SIMD matrix multiplication."""
+
+import random
+import string
+
+import tvm
+from tvm import te
+
+##
+# MxKxN MatMul Intrinsic #
+##
+
+# NOTE this is transposed matmul (A * B^T)
+def intrin_gemm_MxKxN(M, K, N, in_dtype, out_dtype):
+"""Defines a SIMD-accelerated transposed matmul."""
+# we generate a unique ID for every intrinsic definition, to prevent name
+# collisions in the generated source (e.g., if there are multiple operators
+# in the same module that use the same intrinsic)
+#
+# TODO(weberlo, areusch): to cut down on memory usage, we should cache 
each intrinsic
+# instantiation and include it only once, eliminating the need for unique
+# IDs
+UNIQ_ID_LEN = 8
+uniq_id = ''.join(random.choices(string.ascii_uppercase, k=UNIQ_ID_LEN))
+
+if isinstance(M, tvm.tir.IntImm):
+M = M.value
+if isinstance(K, tvm.tir.IntImm):
+K = K.value
+if isinstance(N, tvm.tir.IntImm):
+N = N.value
+assert K % 4 == 0
+# TODO(weberlo, areusch): support more dtypes?
+assert in_dtype == 'int8'
+assert out_dtype == 'int32'
+A = te.placeholder((M, K), name='a', dtype=in_dtype)
+B = te.placeholder((N, K), name='b', dtype=in_dtype)
+k = te.reduce_axis((0, K), name='k')
+C = te.compute(
+(M, N),
+lambda i, j: te.sum(A[i, k].astype(out_dtype) * B[j, 
k].astype(out_dtype), axis=k),
+name='c')
+A_buf = tvm.tir.decl_buffer(
+A.shape, A.dtype,
+name="A",
+offset_factor=1,
+strides=[te.var("A_s"), 1])
+B_buf = tvm.tir.decl_buffer(
+B.shape, B.dtype,
+name="B",
+offset_factor=1,
+strides=[te.var("B_s"), 1])
+C_buf = tvm.tir.decl_buffer(
+C.shape, C.dtype,
+name="C",
+offset_factor=1,
+strides=[te.var("C_s"), 1])
+def intrin_func(ins, outs):
+aa, bb = ins
+cc = outs[0]
+def _reduce_update():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32", 
f"gemm_{M}x{K}x{N}_update_{uniq_id}",
+aa.access_ptr("r"),
+bb.access_ptr("r"),
+cc.access_ptr("w"),
+aa.strides[0],
+bb.strides[0],
+cc.strides[0]))
+return ib.get()
+def _reduce_reset():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32", 
f"gemm_{M}x{K}x{N}_reset_{uniq_id}",
+cc.access_ptr("w"),
+cc.strides[0]))
+return ib.get()
+def _body():
+ib = tvm.tir.ir_builder.create()
+# # NOTE we need the reset in the body for cases where the buffer

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416138909



##
File path: tests/python/unittest/test_runtime_micro.py
##
@@ -25,8 +25,25 @@
 from tvm.micro import create_micro_mod
 from tvm.relay.testing import resnet
 
-# Use the host emulated micro device.
-DEV_CONFIG = micro.device.host.default_config()
+# # Use the host emulated micro device.
+DEV_CONFIG_A = micro.device.host.generate_config()
+DEV_CONFIG_B = micro.device.host.generate_config()
+TARGET = 'c -device=micro_dev'
+
+# # TODO why do spike examples have memory that starts at 0x1000, but you

Review comment:
   i'm not sure what they were there for. it seemed like a thing we would 
hit again, but we can always pull them from the demo tag. deleted.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416138054



##
File path: src/runtime/micro/micro_session.cc
##
@@ -209,59 +246,124 @@ MicroSession::~MicroSession() {
   low_level_device_ = nullptr;
 }
 
-double MicroSession::PushToExecQueue(DevPtr func_ptr, const TVMArgs& args) {
+void MicroSession::PushToTaskQueue(TargetPtr func_ptr, const TVMArgs& args) {
   if (thumb_mode_) {
+// TODO(areusch): should be |=
 func_ptr += 1;
   }
+  TargetVal func_dev_addr = func_ptr.value();
+
+  std::tuple arg_field_addrs = 
EncoderAppend(_args_encoder_, args);
+  TargetVal arg_values_dev_addr{std::get<0>(arg_field_addrs).value()};
+  TargetVal arg_type_codes_dev_addr{std::get<1>(arg_field_addrs).value()};
+
+  task_queue_.push_back(
+  DevTask {
+.func = func_dev_addr,
+.arg_values = arg_values_dev_addr,
+.arg_type_codes = arg_type_codes_dev_addr,
+.num_args = args.num_args
+  });
+
+  if (task_queue_.size() == MicroSession::kTaskQueueCapacity) {
+FlushTaskQueue();
+  }
+}
 
-  // Create an allocator stream for the memory region after the most recent
-  // allocation in the args section.
-  DevPtr args_addr = GetAllocator(SectionKind::kArgs)->curr_end_addr();
-  TargetDataLayoutEncoder encoder(args_addr, word_size_);
-
-  std::tuple arg_field_addrs = EncoderAppend(, args);
-
-  // Flush `stream` to device memory.
-  DevPtr stream_dev_addr =
-  GetAllocator(SectionKind::kArgs)->Allocate(encoder.buf_size());
-  low_level_device()->Write(stream_dev_addr,
-reinterpret_cast(encoder.data()),
-encoder.buf_size());
-
-  TargetVal arg_values_dev_addr = std::get<0>(arg_field_addrs).value();
-  TargetVal arg_type_codes_dev_addr = std::get<1>(arg_field_addrs).value();
-  if (word_size_ == 4) {
-UTVMTask32 task = {
-  .func = func_ptr.value().val32,
-  .arg_values = arg_values_dev_addr.val32,
-  .arg_type_codes = arg_type_codes_dev_addr.val32,
-  .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
-  } else if (word_size_ == 8) {
-UTVMTask64 task = {
-  .func = func_ptr.value().val64,
-  .arg_values = arg_values_dev_addr.val64,
-  .arg_type_codes = arg_type_codes_dev_addr.val64,
-  .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
+void MicroSession::FlushTaskQueue() {
+  if (task_queue_.size() == 0) {
+// nothing to run
+return;
+  }
+  if (word_size_.bytes() == 4) {
+FlushTaskQueuePriv();
+  } else if (word_size_.bytes() == 8) {
+FlushTaskQueuePriv();
   }
+}
 
-  DevPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
-  DevPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
+template 
+void MicroSession::FlushTaskQueuePriv() {
+  // std::cout << "[MicroSession::FlushTaskQueue]" << std::endl;
+  std::vector prepped_tasks;
+  for (const auto& task : task_queue_) {
+prepped_tasks.push_back(T(task));
+  }
+
+  // Flush `args` to device memory.
+  low_level_device()->Write(
+  batch_args_encoder_.start_addr(),
+  reinterpret_cast(batch_args_encoder_.data()),
+  batch_args_encoder_.buf_size());
+
+  // Flush `tasks` to device memory.
+//  runtime_symbol_map_.Dump(std::cout);
+  TargetPtr dev_tasks_addr = runtime_symbol_map_["utvm_tasks"];
+  low_level_device()->Write(
+  dev_tasks_addr,
+  reinterpret_cast(prepped_tasks.data()),
+  prepped_tasks.size() * sizeof(T));
+  DevSymbolWrite(runtime_symbol_map_, "utvm_num_tasks", 
prepped_tasks.size());
+
+  TargetPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
+  TargetPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
   if (thumb_mode_) {
+// TODO(areusch): should be |=
 utvm_init_addr += 1;
   }
 
+  std::chrono::time_point<
+std::chrono::high_resolution_clock, std::chrono::nanoseconds> tbegin, tend;
+  tbegin = std::chrono::high_resolution_clock::now();
+  // std::string tmp;

Review comment:
   these ones in particular might be okay to leave in for now since they 
are helpful if you want to debug. after this, since we want to change the way 
we load code, i'd suggest we improve the debugging story here. but, it will 
change as we move to flashing binaries.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415991766



##
File path: topi/python/topi/arm_cpu/cortex_m7/conv2d/direct.py
##
@@ -0,0 +1,177 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name

Review comment:
   this placement seems to be pretty conventional for `topi`, since most of 
the variable names are too short to be considered valid?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415990923



##
File path: src/target/source/codegen_c_host.cc
##
@@ -20,10 +20,10 @@
 /*!
  * \file codegen_c_host.cc
  */
-#include 
+#include "codegen_c_host.h"

Review comment:
   done, though, can you explain why tvm_header and local_header are 
distinct things in this codebase? does tvm_header mean headers in `include`, 
since they are installed?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415989331



##
File path: src/runtime/micro/micro_common.h
##
@@ -52,28 +53,115 @@ enum class SectionKind : size_t {
   kNumKinds,
 };
 
-/*! \brief union for storing values on varying target word sizes */
-union TargetVal {
-  /*! \brief 32-bit pointer */
-  uint32_t val32;
-  /*! \brief 64-bit pointer */
-  uint64_t val64;
+/*! \brief data type for word sizes */
+class TargetWordSize {
+ private:
+  size_t word_size_bits_;
+
+ public:
+  explicit TargetWordSize(size_t word_size_bits) : 
word_size_bits_{word_size_bits} {
+CHECK(word_size_bits == 32 || word_size_bits == 64)
+  << "only 32-bit and 64-bit are supported now";
+  }
+
+  size_t bytes() const {
+return word_size_bits_ / 8;
+  }
+
+  size_t bits() const {
+return word_size_bits_;
+  }
 };
 
-/*! \brief absolute device address */
-class DevPtr {
+
+/*! \brief class for storing values on varying target word sizes */
+class TargetVal {
+ private:
+  size_t width_bits_;
+  uint64_t value_;
+
  public:
-  /*! \brief construct a device address with value `value` */
-  explicit DevPtr(std::uintptr_t value) : value_(TargetVal { .val64 = value }) 
{}
+  /*! \brief construct a TargetVal matching the size of the given integral 
argument */
+  template::value, T>::type>
+  explicit constexpr TargetVal(T value) : TargetVal(sizeof(T) * 8, value) {}
+
+  /*! \brief construct an uninitialized value */
+  TargetVal() : width_bits_{0}, value_{0} {}
+
+  /*! \brief construct a TargetVal with explicit size and value */
+  TargetVal(size_t width_bits, uint64_t value) : width_bits_{width_bits} {
+CHECK(width_bits >= 8 &&
+  width_bits <= 64 &&
+  (width_bits & (width_bits - 1)) == 0)
+  << "width_bits must be a power of 2 in [8, 64], got " << width_bits;
+value_ = value & bitmask();
+  }
+
+  bool is_initialized() const { return width_bits_ != 0; }

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415988262



##
File path: src/runtime/micro/host_driven/utvm_runtime.h
##
@@ -46,20 +62,38 @@ typedef struct {
   int32_t num_args;
 } UTVMTask;
 
+/*!
+ * \brief TODO
+ */
 extern void UTVMInit();
 
-extern void UTVMTimerReset();
-
+/*!
+ * \brief TODO
+ */
 extern int32_t UTVMTimerStart();
 
-extern void UTVMTimerStop();
-
-extern uint32_t UTVMTimerRead();
+/*!
+ * \brief TODO
+ */
+extern uint32_t UTVMTimerStop(int32_t* err);
 
+/*!
+ * \brief TODO
+ */
 void UTVMMain();
 
+/*!
+ * \brief TODO

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415988618



##
File path: src/runtime/micro/micro_common.h
##
@@ -52,28 +53,115 @@ enum class SectionKind : size_t {
   kNumKinds,
 };
 
-/*! \brief union for storing values on varying target word sizes */
-union TargetVal {
-  /*! \brief 32-bit pointer */
-  uint32_t val32;
-  /*! \brief 64-bit pointer */
-  uint64_t val64;
+/*! \brief data type for word sizes */
+class TargetWordSize {
+ private:
+  size_t word_size_bits_;

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415984080



##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
 
 #include "utvm_runtime.h"
 
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0;  // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };
 
 // These pointers are patched at load time to point to the workspace section.
-char* utvm_workspace_start = NULL;  // NOLINT(*)
-char* utvm_workspace_end = NULL;// NOLINT(*)
-char* utvm_workspace_curr = NULL;   // NOLINT(*)
+volatile char* utvm_workspace_start = NULL;  // NOLINT(*)
+volatile char* utvm_workspace_end = NULL;// NOLINT(*)
+volatile char* utvm_workspace_curr = NULL;   // NOLINT(*)
+#define MAX_WS_ALLOCS 10
+volatile char* utvm_alloc_ends[MAX_WS_ALLOCS] = {};  // NOLINT(*)
+volatile uint32_t utvm_alloc_idx = 0;
 // Keep track of how many active allocations there are on the workspace.
-size_t utvm_num_active_allocs = 0;
+volatile uint32_t utvm_num_active_allocs = 0;
+
+volatile uint32_t utvm_word_size = 0;
 
-const char* utvm_last_error = NULL;  // NOLINT(*)
-int32_t utvm_return_code = 0;// NOLINT(*)
+volatile int32_t utvm_last_error = 0;  // NOLINT(*)
 
-uint32_t utvm_task_time = 0;
+volatile uint32_t utvm_done = 0;
 
 // Gets called by UTVMInit, after device-specific initialization is finished.
 void UTVMMain() {
+  utvm_done = 0;
+  // loss of precision should be fine here, since we only care about the lower 
bits
+  if (((uint32_t) utvm_workspace_start) % utvm_word_size) {
+utvm_last_error = UTVM_ERR_WS_UNALIGNED_START;
+UTVMDone();
+return;
+  }
   utvm_workspace_curr = utvm_workspace_start;
   utvm_num_active_allocs = 0;
-  utvm_last_error = NULL;  // NOLINT(*)
-  utvm_return_code = 0;
-  utvm_task_time = 0;
-  UTVMTimerReset();
-  int32_t err = UTVMTimerStart();
-  if (err < 0) {
-utvm_return_code = err;
-UTVMDone();
+  utvm_alloc_idx = 0;
+  utvm_last_error = UTVM_ERR_NOT_FINISHED;
+  for (uint32_t i = 0; i < utvm_num_tasks; i++) {
+int32_t err = UTVM_ERR_OK;
+utvm_task_times[i] = 0;
+err = UTVMTimerStart();
+if (err < 0) {
+  utvm_last_error = err;
+  UTVMDone();
+  return;
+}
+err = utvm_tasks[i].func(
+(void*) utvm_tasks[i].arg_values,  // NOLINT(*)
+(void*) utvm_tasks[i].arg_type_codes,  // NOLINT(*)
+utvm_tasks[i].num_args);
+if (err < 0) {
+  UTVMDone();
+  return;
+}
+utvm_task_times[i] = UTVMTimerStop();
+if (err < 0) {
+  utvm_last_error = err;
+  UTVMDone();
+  return;
+}
+  }
+  if (utvm_last_error == UTVM_ERR_NOT_FINISHED) {
+utvm_last_error = UTVM_ERR_OK;
   }
-  utvm_return_code = utvm_task.func(
-  (void*) utvm_task.arg_values,  // NOLINT(*)
-  (void*) utvm_task.arg_type_codes,  // NOLINT(*)
-  utvm_task.num_args);
-  UTVMTimerStop();
-  utvm_task_time = UTVMTimerRead();
   UTVMDone();
 }
 
 // We use a dummy function to signal execution is finished for device
 // backends which require breakpoints.
-void UTVMDone() { }
+void __attribute__((noinline)) UTVMDone() {
+  utvm_done = 1;
+}
+
+#define ALIGNED_UP(x, word_size) \
+  word_size) - (((uintptr_t) (x)) % (word_size))) % (word_size)) + (x))
 
 void* TVMBackendAllocWorkspace(int device_type, int device_id, uint64_t size,
int dtype_code_hint, int dtype_bits_hint) {
-  // Align up to 8 bytes.
-  utvm_workspace_curr +=
-(utvm_word_size - ((uintptr_t) utvm_workspace_curr % utvm_word_size)) % 
utvm_word_size;  // NOLINT(*)
-  if (utvm_workspace_curr + size > utvm_workspace_end) {
+  if (size == 0) {
+utvm_last_error = UTVM_ERR_WS_ZERO_SIZE_ALLOC;
+return NULL;
+  }
+  size_t alloc_requested_bytes = size;
+  size_t alloc_size_words = (alloc_requested_bytes + utvm_word_size - 1) / 
utvm_word_size;
+  size_t alloc_size_bytes = alloc_size_words * utvm_word_size;
+
+  // Align up to the target word size.
+  if (utvm_workspace_curr + alloc_size_bytes > utvm_workspace_end) {
 // Out of space in workspace.
+utvm_last_error = UTVM_ERR_WS_OUT_OF_SPACE;
+return NULL;
+  }
+  if (utvm_alloc_idx == MAX_WS_ALLOCS - 1) {
+// Exceeded number of allocs we can keep track of.
+utvm_last_error = UTVM_ERR_WS_TOO_MANY_ALLOCS;
 return NULL;
   }
   void* ret_ptr = (void*) utvm_workspace_curr;  // NOLINT(*)
-  utvm_workspace_curr += size;
+  utvm_workspace_curr = utvm_workspace_curr + alloc_size_bytes;
+  // store the *end* of the alloc, so we can restore the WS pointer when 
freeing
+  

[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415983620



##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
 
 #include "utvm_runtime.h"
 
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0;  // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };
 
 // These pointers are patched at load time to point to the workspace section.
-char* utvm_workspace_start = NULL;  // NOLINT(*)
-char* utvm_workspace_end = NULL;// NOLINT(*)
-char* utvm_workspace_curr = NULL;   // NOLINT(*)
+volatile char* utvm_workspace_start = NULL;  // NOLINT(*)
+volatile char* utvm_workspace_end = NULL;// NOLINT(*)
+volatile char* utvm_workspace_curr = NULL;   // NOLINT(*)
+#define MAX_WS_ALLOCS 10
+volatile char* utvm_alloc_ends[MAX_WS_ALLOCS] = {};  // NOLINT(*)
+volatile uint32_t utvm_alloc_idx = 0;
 // Keep track of how many active allocations there are on the workspace.
-size_t utvm_num_active_allocs = 0;
+volatile uint32_t utvm_num_active_allocs = 0;
+
+volatile uint32_t utvm_word_size = 0;
 
-const char* utvm_last_error = NULL;  // NOLINT(*)
-int32_t utvm_return_code = 0;// NOLINT(*)
+volatile int32_t utvm_last_error = 0;  // NOLINT(*)
 
-uint32_t utvm_task_time = 0;
+volatile uint32_t utvm_done = 0;
 
 // Gets called by UTVMInit, after device-specific initialization is finished.
 void UTVMMain() {
+  utvm_done = 0;
+  // loss of precision should be fine here, since we only care about the lower 
bits
+  if (((uint32_t) utvm_workspace_start) % utvm_word_size) {

Review comment:
   downcasting to a smaller integral type is well-defined, but deciding 
what to do when the cast is missing can be less well-defined (i believe this 
problem is fixed in c++ and potentially newer C). it's fine here. another 
embedded way to fix this is to assume utvm_word_size is a power of two # of 
bytes, and do
   
   `if (utvm_workspace_start & (utvm_word_size - 1))`





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415981837



##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
 
 #include "utvm_runtime.h"
 
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0;  // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };
 
 // These pointers are patched at load time to point to the workspace section.
-char* utvm_workspace_start = NULL;  // NOLINT(*)
-char* utvm_workspace_end = NULL;// NOLINT(*)
-char* utvm_workspace_curr = NULL;   // NOLINT(*)
+volatile char* utvm_workspace_start = NULL;  // NOLINT(*)
+volatile char* utvm_workspace_end = NULL;// NOLINT(*)
+volatile char* utvm_workspace_curr = NULL;   // NOLINT(*)
+#define MAX_WS_ALLOCS 10
+volatile char* utvm_alloc_ends[MAX_WS_ALLOCS] = {};  // NOLINT(*)

Review comment:
   let's leave it as is, since building binaries will likely change how we 
configure most of this





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415981441



##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
 
 #include "utvm_runtime.h"
 
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0;  // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };

Review comment:
   done. we can look at runtime configuration more after we switch to 
building full binaries all the time.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415980487



##
File path: src/runtime/micro/host_driven/utvm_device_dylib_redirect.c
##
@@ -32,10 +32,10 @@ extern "C" {
 #include 
 #include 
 
-void *(*TVMBackendAllocWorkspace_)(int, int, uint64_t, int, int) =
-(void *(*)(int, int, uint64_t, int, int)) NULL;
-int (*TVMBackendFreeWorkspace_)(int, int, void*) = (int (*)(int, int, void*)) 
NULL;
-void (*TVMAPISetLastError_)(const char*) = (void (*)(const char*)) NULL;
+// TODO(areusch): compiler errors say volatile qualifier is discarded. should 
we just get rid of em?

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415970964



##
File path: src/ir/error.cc
##
@@ -62,16 +62,21 @@ void ErrorReporter::RenderErrors(const IRModule& module, 
bool use_color) {
 
 CHECK(has_errs != this->node_to_error_.end());
 
-const auto& error_indicies = has_errs->second;
+const auto& error_indices = has_errs->second;
 
 std::stringstream err_msg;
 
-err_msg << rang::fg::red;
-err_msg << " ";
-for (auto index : error_indicies) {
-  err_msg << this->errors_[index].what() << "; ";
+if (error_indices.size() != 0) {
+  err_msg << rang::fg::red;
+  err_msg << " ";
+  // the errors are in reverse order, so print them with a reversed 
iteration
+  err_msg << this->errors_[error_indices[error_indices.size()-1]].what();
+  for (int i = error_indices.size() - 2; i >= 0; i--) {
+size_t err_idx = error_indices[i];
+err_msg << "; " << this->errors_[err_idx].what();
+  }
+  err_msg << rang::fg::reset;

Review comment:
   reverted





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415970450



##
File path: python/tvm/target/arm_isa.py
##
@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Defines functions to analyze availble opcodes in the ARM ISA."""

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415967381



##
File path: python/tvm/relay/op/strategy/arm_cpu.py
##
@@ -267,7 +281,7 @@ def bitserial_conv2d_strategy_arm_cpu(attrs, inputs, 
out_type, target):
 raise ValueError("Data layout {} not supported.".format(layout))
 return strategy
 
-@bitserial_dense_strategy.register("arm_cpu")
+@bitserial_dense_strategy.register(["arm_cpu", "micro_dev"])

Review comment:
   oh good catch, reverted. yes we are punting on that for a bit.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415961064



##
File path: python/tvm/relay/_parser.py
##
@@ -340,7 +342,10 @@ def visitLocalVar(self, ctx):
 return local_var
 
 def visitGraphVar(self, ctx):
-return self.graph_expr[int(ctx.NAT().getText())]
+graph_var_idx = int(ctx.NAT().getText())
+if graph_var_idx >= len(self.graph_expr):
+raise ParseError(f"graph var `%{graph_var_idx}` is unbound")
+return self.graph_expr[graph_var_idx]

Review comment:
   ok, what about the changes on line 154 as well? I don't know what they 
do yet.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415959267



##
File path: python/tvm/micro/device/host.py
##
@@ -17,12 +17,26 @@
 """Compilation and config definitions for the host emulated device"""
 import sys
 
-from . import create_micro_lib_base, register_device
+from . import create_micro_lib_base, register_device, gen_mem_layout, 
MemConstraint
 
-DEVICE_ID = "host"
-TOOLCHAIN_PREFIX = ""
+DEVICE_ID = 'host'

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415958851



##
File path: python/tvm/micro/device/base.py
##
@@ -92,71 +95,142 @@ def create_micro_lib_base(
 
 options : List[str]
 additional options to pass to GCC
+
+lib_src_paths : Optional[List[str]]
+paths to additional source files to be compiled into the library
 """
+# look at these (specifically `strip`):
+#   
https://stackoverflow.com/questions/15314581/g-compiler-flag-to-minimize-binary-size
 base_compile_cmd = [
-f"{toolchain_prefix}gcc",
-"-std=c11",
-"-Wall",
-"-Wextra",
-"--pedantic",
-"-c",
-"-O0",
-"-g",
-"-nostartfiles",
-"-nodefaultlibs",
-"-nostdlib",
-"-fdata-sections",
-"-ffunction-sections",
+f'{toolchain_prefix}gcc',
+'-std=c11',
+'-Wall',
+'-Wextra',
+'--pedantic',
+'-c',
+'-g',
+'-nostartfiles',
+'-nodefaultlibs',
+'-nostdlib',
+'-fdata-sections',
+'-ffunction-sections',
 ]
 if options is not None:
 base_compile_cmd += options
 
 src_paths = []
 include_paths = find_include_path() + [get_micro_host_driven_dir()]
 tmp_dir = _util.tempdir()
-# we might transform the src path in one of the branches below
+# we need to create a new src file in the operator branch
 new_in_src_path = in_src_path
 if lib_type == LibType.RUNTIME:
 dev_dir = _get_device_source_dir(device_id)
-dev_src_paths = glob.glob(f"{dev_dir}/*.[csS]")
+
+dev_src_paths = glob.glob(f'{dev_dir}/*.[csS]')
 # there needs to at least be a utvm_timer.c file
 assert dev_src_paths
-assert "utvm_timer.c" in map(os.path.basename, dev_src_paths)
+assert 'utvm_timer.c' in map(os.path.basename, dev_src_paths)
+
 src_paths += dev_src_paths
 elif lib_type == LibType.OPERATOR:
-# create a temporary copy of the source, so we can inject the dev lib
+# create a temporary copy of the operator source, so we can inject the 
dev lib
 # header without modifying the original.
-temp_src_path = tmp_dir.relpath("temp.c")
-with open(in_src_path, "r") as f:
+temp_src_path = tmp_dir.relpath('temp.c')
+with open(in_src_path, 'r') as f:
 src_lines = f.read().splitlines()
-src_lines.insert(0, "#include \"utvm_device_dylib_redirect.c\"")
-with open(temp_src_path, "w") as f:
-f.write("\n".join(src_lines))
+src_lines.insert(0, '#include "utvm_device_dylib_redirect.c"')
+with open(temp_src_path, 'w') as f:
+f.write('\n'.join(src_lines))
 new_in_src_path = temp_src_path
-base_compile_cmd += ["-c"]
 else:
-raise RuntimeError("unknown lib type")
+raise RuntimeError('unknown lib type')
 
 src_paths += [new_in_src_path]
 
+# add any src paths required by the operator
+if lib_src_paths is not None:
+src_paths += lib_src_paths
+
+# print(f'include paths: {include_paths}')
 for path in include_paths:
-base_compile_cmd += ["-I", path]
+base_compile_cmd += ['-I', path]

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415958560



##
File path: python/tvm/micro/device/base.py
##
@@ -92,71 +95,142 @@ def create_micro_lib_base(
 
 options : List[str]
 additional options to pass to GCC
+
+lib_src_paths : Optional[List[str]]
+paths to additional source files to be compiled into the library
 """
+# look at these (specifically `strip`):
+#   
https://stackoverflow.com/questions/15314581/g-compiler-flag-to-minimize-binary-size
 base_compile_cmd = [
-f"{toolchain_prefix}gcc",
-"-std=c11",
-"-Wall",
-"-Wextra",
-"--pedantic",
-"-c",
-"-O0",
-"-g",
-"-nostartfiles",
-"-nodefaultlibs",
-"-nostdlib",
-"-fdata-sections",
-"-ffunction-sections",
+f'{toolchain_prefix}gcc',

Review comment:
   undid all the quote conversions, except those that made sense (i.e. to 
avoid escapes, as in f'unknown variable "{var}"'). can you point out any other 
changes you want me to revert? I didn't originally author most of this code so 
I don't have all the context in my head.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415949272



##
File path: python/tvm/autotvm/tuner/tuner.py
##
@@ -150,7 +150,15 @@ def tune(self, n_trial, measure_option, 
early_stopping=None, callbacks=(), si_pr
  i + k + 1, si_prefix, format_si_prefix(flops, 
si_prefix),
  format_si_prefix(self.best_flops, si_prefix), 
res, config)
 
-i += len(results)
+num_successes = 0
+for result in results:
+if isinstance(result.costs[0], float):
+num_successes += 1
+if num_successes != len(results):
+logger.debug('not counting %d failures towards trial count',
+ len(results) - num_successes)
+i += num_successes
+

Review comment:
   hmm okay. we do need to improve AutoTVM error handling. @tqchen what is 
best to do here?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415947706



##
File path: python/tvm/micro/device/arm/stm32f746xx.py
##
@@ -36,23 +55,41 @@ def create_micro_lib(obj_path, src_path, lib_type, 
options=None):
 
 options : Optional[List[str]]
 additional options to pass to GCC
+
+lib_src_paths : Optional[List[str]]
+TODO
 """
 if options is None:
 options = []
+else:
+options = list(options)
+
 options += [
-"-mcpu=cortex-m7",
-"-mlittle-endian",
-"-mfloat-abi=hard",
-"-mfpu=fpv5-sp-d16",
-"-mthumb",
-"-gdwarf-5",
+# TODO(weberlo): make a debug flag
+'-O2',
+'-march=armv7e-m',
+'-mcpu=cortex-m7',
+'-mlittle-endian',
+'-mfloat-abi=hard',
+'-mfpu=fpv5-sp-d16',
+'-mthumb',
+'-ffast-math',
+'-gdwarf-5',
+'-DARM_MATH_CM7',
+'-D__FPU_PRESENT=1U',
+'-DARM_MATH_DSP',
+'-Wno-unused-variable',

Review comment:
   these are in the generated GEMM code. for example:
   
   `/var/folders/9y/3j808g591ln3kys4qpyl3qmcgn/T/tmpb4sk1ylf/temp.c:104:11: 
error: unused variable \'arg1_code\' [-Werror=unused-variable]`
   
   it would be nice if we did not need to include this, but i'm hoping we can 
merge this PR as-is and incrementally improve things like this (especially 
since I didn't author most of the code generation stuff and will need a bit of 
time to understand how to improve it to remove errors like this). the next PR 
will look at the generated code more in detail, so if it's a quick fix, I can 
fix  it then.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-27 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415942734



##
File path: Makefile
##
@@ -73,7 +73,10 @@ build/libtvm_web_runtime.js: build/libtvm_web_runtime.bc
 cpplint:
python3 3rdparty/dmlc-core/scripts/lint.py vta cpp vta/include vta/src
python3 3rdparty/dmlc-core/scripts/lint.py topi cpp topi/include;
-   python3 3rdparty/dmlc-core/scripts/lint.py tvm cpp include src \
+   # Note: exclude src/runtime/micro/host_driven becuase it contains C99 
files.
+   python3 3rdparty/dmlc-core/scripts/lint.py tvm cpp \
+--exclude_path=src/runtime/micro/host_driven \

Review comment:
   done. in the long term, we need to figure out whether we extend the dmlc 
lint script to not recognize .c files as c++ (this is why I originally excluded 
it). for now, the errors are only related to casting so we can leave it on.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414956089



##
File path: python/tvm/contrib/binutil.py
##
@@ -220,17 +228,32 @@ def tvm_callback_relocate_binary(
 stack_pointer_init=stack_pointer_init)
 
 tmp_dir = util.tempdir()
-rel_obj_path = tmp_dir.relpath("relocated.obj")
-rel_ld_script_path = tmp_dir.relpath("relocated.lds")
-with open(rel_ld_script_path, "w") as f:
+rel_obj_path = tmp_dir.relpath('relocated.obj')
+rel_ld_script_path = tmp_dir.relpath('relocate.lds')
+with open(rel_ld_script_path, 'w') as f:
 f.write(ld_script_contents)
 run_cmd([
-"{}ld".format(toolchain_prefix),
+'{}ld'.format(toolchain_prefix),
 binary_path,
-"-T", rel_ld_script_path,
-"-o", rel_obj_path])
-with open(rel_obj_path, "rb") as f:
+'-T', rel_ld_script_path,
+'-o', rel_obj_path])
+
+with open(rel_obj_path, 'rb') as f:
 rel_bin = bytearray(f.read())
+
+gdb_init_dir = os.environ.get('MICRO_GDB_INIT_DIR')
+if gdb_init_dir is not None:
+gdb_init_path = f'{gdb_init_dir}/.gdbinit'
+with open(gdb_init_path, 'r') as f:
+gdbinit_contents = f.read().split('\n')
+new_contents = []
+for line in gdbinit_contents:
+new_contents.append(line)
+if line.startswith('target'):
+new_contents.append(f'add-symbol-file {rel_obj_path}')
+with open(gdb_init_path, 'w') as f:
+f.write('\n'.join(new_contents))

Review comment:
   I think that's also going to change soon, so would prefer to fix then





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414955744



##
File path: python/tvm/micro/base.py
##
@@ -133,44 +152,91 @@ def __exit__(self, exc_type, exc_value, exc_traceback):
 self._exit()
 
 
-def create_micro_mod(c_mod, dev_config):
+def _calc_max_workspace_usage(src):
+# TODO factor in alignment to the calculation (alloc sizes will be aligned 
up to the word size)
+alloc_re = re.compile(
+r'.*\* ?(.+) = (\(.+\))? TVMBackendAllocWorkspace\(.+, .+, 
\(uint64_t\)(.+), .+, .+\).*')
+free_re = re.compile(r'.*if \(TVMBackendFreeWorkspace\(.+, .+, 
(\(void\*\))? (.+)\) != 0\) {.*')
+max_usage = 0
+alloc_map = {}
+for line in src.split('\n'):
+if line.strip().startswith('//'):
+continue
+match = alloc_re.match(line)
+if match is not None:
+alloc_map[match.group(1)] = int(match.group(3))
+max_usage = max(max_usage, sum(alloc_map.values()))
+else:
+match = free_re.match(line)
+if match is not None:
+print(alloc_map)
+del alloc_map[match.group(2)]
+return max_usage
+
+

Review comment:
   yeah more robust memory analysis will come in a follow-on. can you 
explain what you mean by crash if it doesn't match the format given? I don't 
know it will crash if it finds 0 allocs, it will just not catch anything





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414955794



##
File path: python/tvm/micro/device/host.py
##
@@ -38,59 +52,65 @@ def create_micro_lib(obj_path, src_path, lib_type, 
options=None):
 
 options : Optional[List[str]]
 additional options to pass to GCC
+
+lib_src_paths : Optional[List[str]]
+paths to additional source files to be compiled into the library
 """
 if options is None:
 options = []
-if sys.maxsize > 2**32 and sys.platform.startswith("linux"):
-options += ["-mcmodel=large"]
+else:
+options = list(options)
+# Cannot increase optimization level on host due to code loading method.
+options.append('-O0')

Review comment:
   let's just leave as is for now--the runtime will change a lot soon





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414955335



##
File path: python/tvm/contrib/debugger/debug_runtime.py
##
@@ -181,6 +181,7 @@ def _run_debug(self):
 """
 self.debug_datum._time_list = [
 [float(t) * 1e-6] for t in self.run_individual(10, 1, 1)
+#[float(t) * 1e-6] for t in self.run_individual(1, 1, 1)

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414919556



##
File path: topi/python/topi/arm_cpu/cortex_m7/micro_kernel/gemm.py
##
@@ -0,0 +1,221 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, no-value-for-parameter
+"""Defines gemm intrinsics for SIMD matrix multiplication."""
+
+import random
+import string
+
+import tvm
+from tvm import te
+
+##
+# MxKxN MatMul Intrinsic #
+##
+
+# NOTE this is transposed matmul (A * B^T)
+def intrin_gemm_MxKxN(M, K, N, in_dtype, out_dtype):
+"""Defines a SIMD-accelerated transposed matmul."""
+# we generate a unique ID for every intrinsic definition, to prevent name
+# collisions in the generated source (e.g., if there are multiple operators
+# in the same module that use the same intrinsic)
+#
+# TODO to cut down on memory usage, we should cache each intrinsic
+# instantiation and include it only once, eliminating the need for unique
+# IDs
+UNIQ_ID_LEN = 8
+uniq_id = ''.join(random.choices(string.ascii_uppercase, k=UNIQ_ID_LEN))
+
+if isinstance(M, tvm.tir.IntImm):
+M = M.value
+if isinstance(K, tvm.tir.IntImm):
+K = K.value
+if isinstance(N, tvm.tir.IntImm):
+N = N.value
+assert K % 4 == 0
+# TODO support more dtypes?
+assert in_dtype == 'int8'
+assert out_dtype == 'int32'
+A = te.placeholder((M, K), name='a', dtype=in_dtype)
+B = te.placeholder((N, K), name='b', dtype=in_dtype)
+k = te.reduce_axis((0, K), name='k')
+C = te.compute(
+(M, N),
+lambda i, j: te.sum(A[i, k].astype(out_dtype) * B[j, 
k].astype(out_dtype), axis=k),
+name='c')
+A_buf = tvm.tir.decl_buffer(
+A.shape, A.dtype,
+name="A",
+offset_factor=1,
+strides=[te.var("A_s"), 1])
+B_buf = tvm.tir.decl_buffer(
+B.shape, B.dtype,
+name="B",
+offset_factor=1,
+strides=[te.var("B_s"), 1])
+C_buf = tvm.tir.decl_buffer(
+C.shape, C.dtype,
+name="C",
+offset_factor=1,
+strides=[te.var("C_s"), 1])
+def intrin_func(ins, outs):
+aa, bb = ins
+cc = outs[0]
+def _reduce_update():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32", 
f"gemm_{M}x{K}x{N}_update_{uniq_id}",
+aa.access_ptr("r"),
+bb.access_ptr("r"),
+cc.access_ptr("w"),
+aa.strides[0],
+bb.strides[0],
+cc.strides[0]))
+return ib.get()
+def _reduce_reset():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32", 
f"gemm_{M}x{K}x{N}_reset_{uniq_id}",
+cc.access_ptr("w"),
+cc.strides[0]))
+return ib.get()
+def _body():
+ib = tvm.tir.ir_builder.create()
+# # NOTE we need the reset in the body for cases where the buffer
+# # we're accumulating into is uninitialized (e.g., if it's the
+# # result of a workspace allocation, because there are no 
guarantees
+# # on the contents).
+# ib.emit(tvm.tir.call_extern("int32", f"gemm_{M}x{K}x{N}_reset",
+# cc.access_ptr("w"),
+# cc.strides[0]))
+# ib.emit(tvm.tir.call_extern("int32", f"gemm_{M}x{K}x{N}_update",
+# aa.access_ptr("r"),
+# bb.access_ptr("r"),
+# cc.access_ptr("w"),
+# aa.strides[0],
+# bb.strides[0],
+# cc.strides[0]))
+ib.emit(tvm.tir.call_extern("int32", 
f"gemm_{M}x{K}x{N}_body_{uniq_id}",
+  

[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414919196



##
File path: tests/python/unittest/test_runtime_micro.py
##
@@ -25,8 +25,25 @@
 from tvm.micro import create_micro_mod
 from tvm.relay.testing import resnet
 
-# Use the host emulated micro device.
-DEV_CONFIG = micro.device.host.default_config()
+# # Use the host emulated micro device.
+DEV_CONFIG_A = micro.device.host.generate_config()
+DEV_CONFIG_B = micro.device.host.generate_config()
+TARGET = 'c -device=micro_dev'

Review comment:
   we still have test_interleave_sessions for now though?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414919333



##
File path: topi/python/topi/arm_cpu/conv2d_spatial_pack.py
##
@@ -173,7 +173,7 @@ def schedule_conv2d_spatial_pack_nchw(cfg, s, data_vec, 
kernel_vec,
  axis_lens=[cfg['tile_oh'].size[-1],
 cfg['tile_ow'].size[-1],
 cfg['tile_co'].size[-1]],
- max_unroll=16,
+ max_unroll=None,

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414918682



##
File path: src/runtime/micro/openocd_low_level_device.cc
##
@@ -210,9 +210,9 @@ class OpenOCDLowLevelDevice final : public LowLevelDevice {
   // NOTE: OpenOCD will call any request larger than this constant an "absurd
   // request".
   /*! \brief maximum number of bytes allowed in a single memory transfer */
-  static const constexpr ssize_t kMemTransferLimit = 64000;
+  static const constexpr ssize_t kMemTransferLimit = 8000;

Review comment:
   I can't remember anymore exactly where this limit hits, but iirc it's 
due to mac os x pipe buffering. I think it's because we are reading the pipe 
line by line on the TVM side, but if you issue a memory transfer that prints 
more than ~24k of characters, the os pipe buffer fills up before the newline 
char is sent and we deadlock. updated comment.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7

2020-04-24 Thread GitBox


areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r414918275



##
File path: src/runtime/micro/micro_session.cc
##
@@ -489,6 +629,16 @@ PackedFunc MicroSession::GetFunction(
 return PackedFunc([sptr_to_self](TVMArgs args, TVMRetValue* rv) {
   MicroSession::ExitWithScope();
 });
+// TODO(weberlo): add a `clear_batch_timer` func
+  } else if (name == "get_last_batch_time") {
+return PackedFunc([sptr_to_self, this](TVMArgs args, TVMRetValue* rv) {
+  *rv = this->GetLastBatchTime();
+});
+// TODO(weberlo): remove this func
+  } else if (name == "get_last_batch_cycles") {
+return PackedFunc([sptr_to_self, this](TVMArgs args, TVMRetValue* rv) {
+  *rv = this->GetLastBatchCycles();
+});

Review comment:
   right now it returns either last host or last device time based on 
`use_device_timer`. I agree we should rethink the timing API, but perhaps we 
can move it to next PR, when I would want to implement an API between the host 
and device, and we can better define the concept of batch time (and the units 
it is logged in) there?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org