samskalicky commented on a change in pull request #15886: Graph Partition API
URL: https://github.com/apache/incubator-mxnet/pull/15886#discussion_r317861028
##########
File path: src/c_api/c_api_symbolic.cc
##########
@@ -1199,3 +1200,73 @@ int MXShallowCopySymbol(SymbolHandle src, SymbolHandle*
out) {
*out = out_sym;
API_END_HANDLE_ERROR(delete out_sym);
}
+
+int MXOptimizeForBackend(SymbolHandle sym_handle,
+ const char* backend_name,
+ SymbolHandle* ret_sym_handle,
+ const mx_uint len,
+ NDArrayHandle* in_args_handle,
+ const mx_uint num_options,
+ const char** keys,
+ const char** vals) {
+ nnvm::Symbol *s = new nnvm::Symbol();
+ API_BEGIN();
+ nnvm::Symbol *sym = static_cast<nnvm::Symbol *>(sym_handle);
+ *s = sym->Copy();
+ nnvm::Graph g = Symbol2Graph(*s);
+ if (len) {
+ NDArray **in_args_ptr = reinterpret_cast<NDArray**>(in_args_handle);
+ Context default_ctx = in_args_ptr[0]->ctx();
+ mxnet::ShapeVector arg_shapes(len);
+ nnvm::DTypeVector arg_dtypes(len);
+ StorageTypeVector arg_stypes(len);
+ for (mx_uint i = 0; i < len; i++) {
+ const auto &in_arg = *(in_args_ptr[i]);
+ arg_shapes[i] = in_arg.shape();
+ arg_dtypes[i] = in_arg.dtype();
+ arg_stypes[i] = in_arg.storage_type();
+ CHECK(in_arg.ctx() == default_ctx)
+ << "args[" << i << "] is on context: " << in_arg.ctx()
+ << ", whereas args[0] is on context: " << default_ctx
+ << ". All args must be on the same context.";
+ }
+ const auto& indexed_graph = g.indexed_graph();
+ const auto num_forward_inputs = indexed_graph.input_nodes().size();
+ g.attrs["context"] = std::make_shared<nnvm::any>(
+ exec::ContextVector(indexed_graph.num_nodes(), default_ctx));
+ // infer shapes
+ g = exec::InferShape(std::move(g), std::move(arg_shapes), "__shape__");
Review comment:
Currently these APIs are called from multiple places, here are a few:
https://github.com/apache/incubator-mxnet/blob/d8b6e47b91a737d29c0589bda8640aee69d8599b/src/executor/graph_executor.cc#L460-L478
https://github.com/apache/incubator-mxnet/blob/d8b6e47b91a737d29c0589bda8640aee69d8599b/src/executor/graph_executor.cc#L825-L841
https://github.com/apache/incubator-mxnet/blob/d8b6e47b91a737d29c0589bda8640aee69d8599b/src/executor/graph_executor.cc#L900-L903
https://github.com/apache/incubator-mxnet/blob/d8b6e47b91a737d29c0589bda8640aee69d8599b/src/executor/graph_executor.cc#L1396-L1467
https://github.com/apache/incubator-mxnet/blob/d8b6e47b91a737d29c0589bda8640aee69d8599b/src/executor/graph_executor.cc#L117
So I dont think we need to push to not create another one. Specifically its
saving a lot of trouble for not creating: ctx_map, in_arg_ctxes,
aux_state_ctxes and then parsing then in AssignContext just so that we can set
the "context" attribute on the graph:
https://github.com/apache/incubator-mxnet/blob/6a8d9eb5fd4f7133c094149dc80a3a236534f223/src/common/exec_utils.h#L512-L513
I agree it would be nice to unify all the callers across the codebase. but I
dont think it makes sense to overload this PR with that too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
