samskalicky commented on a change in pull request #15886: Graph Partition API
URL: https://github.com/apache/incubator-mxnet/pull/15886#discussion_r318703729
##########
File path: src/c_api/c_api_symbolic.cc
##########
@@ -1199,3 +1200,73 @@ int MXShallowCopySymbol(SymbolHandle src, SymbolHandle*
out) {
*out = out_sym;
API_END_HANDLE_ERROR(delete out_sym);
}
+
+int MXOptimizeForBackend(SymbolHandle sym_handle,
+ const char* backend_name,
+ SymbolHandle* ret_sym_handle,
+ const mx_uint len,
+ NDArrayHandle* in_args_handle,
+ const mx_uint num_options,
+ const char** keys,
+ const char** vals) {
+ nnvm::Symbol *s = new nnvm::Symbol();
+ API_BEGIN();
+ nnvm::Symbol *sym = static_cast<nnvm::Symbol *>(sym_handle);
+ *s = sym->Copy();
+ nnvm::Graph g = Symbol2Graph(*s);
+ if (len) {
+ NDArray **in_args_ptr = reinterpret_cast<NDArray**>(in_args_handle);
+ Context default_ctx = in_args_ptr[0]->ctx();
Review comment:
First, lets clarify that we already help users avoid this problem by not
explicitly accepting a ctx argument to optimize_for. Instead we'll pull the
context from where the args reside.
https://github.com/apache/incubator-mxnet/blob/9ccf6c60f38b1589595d4d5f6d31d00b846150dd/src/c_api/c_api_symbolic.cc#L1219-L1232
We're not trying to support multi-context execution yet.
We checked and theres no way to "mark" or set an attribute on the
partitioned symbol that we return. So for now we cannot enforce the requirement
that the same context be used for subsequent calls to optimize_for or calls to
bind.
But we expect to have optimization passes that could be context-agnostic.
And some that are context-specific. So the way MXNet currently enforces this is
by having context-specific FCompute functions for operators. This just means
that context-specific optimizations should insert operators that only have an
FCompute function registered for that context.
So multiple calls to optimize_for with different contexts would succeed. But
at runtime executing the subgraph operator on a different context would fail.
Thats the current and only way to enforce this. So the requirement will be that
the subgraph property inserts subgraph operators that only support that context
by only defining FCompute for the desired context.
This is already implemented:
https://github.com/apache/incubator-mxnet/blob/master/src/operator/subgraph/mkldnn/mkldnn_conv.cc#L780
or
https://github.com/apache/incubator-mxnet/blob/master/src/operator/subgraph/tensorrt/tensorrt.cu#L64
So if we ran the following example:
```
sym = sym.optimize_for('MKLDNN', ctx= mx.cpu(), args)
sym = sym.optimize_for('TensorRT', ctx= mx.gpu(), args)
```
the calls to optimize_for would succeed. Presumably there will be both
mkldnn and tensorrt subgraph ops inserted. But at runtime the user will see an
error about one or the other not being supported on that context.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
