[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551682259



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -303,12 +315,28 @@ class LocalBuilder(ProgramBuilder):
 This is used in a wrapper of the multiprocessing.Process.join().
 n_parallel : int = multiprocessing.cpu_count()
 Number of threads used to build in parallel.
-build_func : str = 'default'
-The name of registered build function.
+build_func: callable or str = "default"
+If is 'default', use default build function
+If is 'ndk', use function for android ndk
+If is callable, use it as custom build function, expect lib_format 
field.
 """
 
 def __init__(self, timeout=15, n_parallel=multiprocessing.cpu_count(), 
build_func="default"):
-self.__init_handle_by_constructor__(_ffi_api.LocalBuilder, timeout, 
n_parallel, build_func)
+if build_func == "default":
+BuildFunc.name = "default"
+BuildFunc.build_func = tar.tar
+elif build_func == "ndk":
+BuildFunc.name = "ndk"
+BuildFunc.build_func = ndk.create_shared
+elif not isinstance(build_func, str):
+BuildFunc.name = "custom"
+BuildFunc.build_func = build_func

Review comment:
   OK, I have already modified





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551207425



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   `
   def cross_compiler(
   compile_func, options=None, output_format=None, get_target_triple=None, 
add_files=None
   ):
   def _fcompile(outputs, objects, options=None):
   all_options = base_options
   if options is not None:
   all_options += options
   compile_func(outputs, objects + add_files, options=all_options, 
**kwargs)
   
   if not output_format and hasattr(compile_func, "output_format"):
   output_format = compile_func.output_format
   output_format = output_format if output_format else "so"
   
   if not get_target_triple and hasattr(compile_func, "get_target_triple"):
   get_target_triple = compile_func.get_target_triple
   
   _fcompile.output_format = output_format
   _fcompile.get_target_triple = get_target_triple
   return _fcompile
   `





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551207425



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   `
   def cross_compiler(
   compile_func, options=None, output_format=None, get_target_triple=None, 
add_files=None
   ):
   
   def _fcompile(outputs, objects, options=None):
   all_options = base_options
   if options is not None:
   all_options += options
   compile_func(outputs, objects + add_files, options=all_options, 
**kwargs)
   
   if not output_format and hasattr(compile_func, "output_format"):
   output_format = compile_func.output_format
   output_format = output_format if output_format else "so"
   
   if not get_target_triple and hasattr(compile_func, "get_target_triple"):
   get_target_triple = compile_func.get_target_triple
   
   _fcompile.output_format = output_format
   _fcompile.get_target_triple = get_target_triple
   return _fcompile
   `





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551207425



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   `
   def cross_compiler(
   compile_func, options=None, output_format=None, get_target_triple=None, 
add_files=None
   ):
   ...
   
   def _fcompile(outputs, objects, options=None):
   all_options = base_options
   if options is not None:
   all_options += options
   compile_func(outputs, objects + add_files, options=all_options, 
**kwargs)
   
   if not output_format and hasattr(compile_func, "output_format"):
   output_format = compile_func.output_format
   output_format = output_format if output_format else "so"
   
   if not get_target_triple and hasattr(compile_func, "get_target_triple"):
   get_target_triple = compile_func.get_target_triple
   
   _fcompile.output_format = output_format
   _fcompile.get_target_triple = get_target_triple
   return _fcompile
   `





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551195931



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   yes





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   @jcf94 
   I try use @tvm._ffi.register_func, but the custom build func is a python 
callable, it is not just function name. we need the whole python callable 
context,  and the callable also have attribute. like cross_compiler, it has 
output_format and get_target_triple attributes.
   _
   use_ndk = True
   build_func = cc.cross_compiler(ndk.create_shared, options=ndk_options)
   tuner = auto_scheduler.TaskScheduler(tasks, task_weights)
   tune_option = auto_scheduler.TuningOptions(
   num_measure_trials=200,  # change this to 2 to achieve the best 
performance
   builder=auto_scheduler.LocalBuilder(build_func=build_func if use_ndk 
else "default"),
   runner=auto_scheduler.RPCRunner(
   device_key, host=tracker_ip, port=tracker_port, repeat=3, 
timeout=50
   ),
   measure_callbacks=[auto_scheduler.RecordToFile(log_file)],
   )
   _





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   @jcf94 
   I try use @tvm._ffi.register_func, but the custom build func is a python 
callable, it is not just function name. we need the whole python callable 
context,  and the callable also have attribute. like cross_compiler, it has 
output_format and get_target_triple attributes.
   use_ndk = True
   build_func = cc.cross_compiler(ndk.create_shared, options=ndk_options)
   tuner = auto_scheduler.TaskScheduler(tasks, task_weights)
   tune_option = auto_scheduler.TuningOptions(
   num_measure_trials=200,  # change this to 2 to achieve the best 
performance
   builder=auto_scheduler.LocalBuilder(build_func=build_func if use_ndk 
else "default"),
   runner=auto_scheduler.RPCRunner(
   device_key, host=tracker_ip, port=tracker_port, repeat=3, 
timeout=50
   ),
   measure_callbacks=[auto_scheduler.RecordToFile(log_file)],
   )





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   @jcf94 
   I try use @tvm._ffi.register_func, but the custom build func is a python 
callable, it is not just function. we need the whole python callable context,  
   build_func = cc.cross_compiler(ndk.create_shared, options=ndk_options)





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-04 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +652,10 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+assert build_func == BuildFunc.name, (
+"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func
+)
+build_func = BuildFunc.build_func

Review comment:
   @jcf94 
   I try use @tvm._ffi.register_func, but the custom build func is a python 
callable, it is not just function. we need the whole python callable context, 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-03 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551162129



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -624,12 +647,7 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-if build_func == "default":
-build_func = tar.tar
-elif build_func == "ndk":
-build_func = ndk.create_shared
-else:
-raise ValueError("Invalid build_func" + build_func)
+build_func = BuildFunc.build_func

Review comment:
   custom build func is a python callable object, and it has attribute and 
so on, we can not serialize it into args.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-03 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551153793



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -652,7 +652,6 @@ def local_build_worker(args):
 The build result of this Builder thread.
 """
 inp, build_func, timeout, verbose = args
-assert any(build_func == name for name in BuildFunc.name)

Review comment:
   any is error, so I change it





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-03 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551140443



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -303,12 +312,26 @@ class LocalBuilder(ProgramBuilder):
 This is used in a wrapper of the multiprocessing.Process.join().
 n_parallel : int = multiprocessing.cpu_count()
 Number of threads used to build in parallel.
-build_func : str = 'default'
-The name of registered build function.
+build_func: callable or str = "default"
+If is 'default', use default build function
+If is 'ndk', use function for android ndk
+If is callable, use it as custom build function, expect lib_format 
field.
 """
 
 def __init__(self, timeout=15, n_parallel=multiprocessing.cpu_count(), 
build_func="default"):
-self.__init_handle_by_constructor__(_ffi_api.LocalBuilder, timeout, 
n_parallel, build_func)
+if build_func == "default":

Review comment:
   when auto_scheduler.local_builder.build is called, the context can't get 
LocalBuilder class instance





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-03 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551135097



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -63,6 +63,9 @@
 # We use 1e10 instead of sys.float_info.max for better readability in log
 MAX_FLOAT = 1e10
 
+class CustomBuildFunc:

Review comment:
   done
   





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-03 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551133295



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -303,11 +306,16 @@ class LocalBuilder(ProgramBuilder):
 This is used in a wrapper of the multiprocessing.Process.join().
 n_parallel : int = multiprocessing.cpu_count()
 Number of threads used to build in parallel.
-build_func : str = 'default'
-The name of registered build function.
+build_func: callable or str

Review comment:
   I have added BuildFunc default name = "default" and default func 
callable = tar.tar





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function

2021-01-03 Thread GitBox 


leowang1225 commented on a change in pull request #7185:
URL: https://github.com/apache/tvm/pull/7185#discussion_r551132967



##
File path: python/tvm/auto_scheduler/measure.py
##
@@ -628,6 +636,8 @@ def local_build_worker(args):
 build_func = tar.tar
 elif build_func == "ndk":
 build_func = ndk.create_shared
+elif build_func == "custom":

Review comment:
   I change all conditions to BuildFunc





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org