eric-haibin-lin commented on a change in pull request #14200: Bulked op
segments to allow Variable nodes
URL: https://github.com/apache/incubator-mxnet/pull/14200#discussion_r261845325
##########
File path: src/executor/graph_executor.cc
##########
@@ -1211,63 +1212,53 @@ void GraphExecutor::InitOpSegs() {
void GraphExecutor::BulkTrainingOpSegs(size_t total_num_nodes) {
- // The maximum number of node in a segment executed in bulk
- size_t num_nodes_threshold =
dmlc::GetEnv("MXNET_EXEC_BULK_EXEC_MAX_NODE_TRAIN", 15);
+ // The maximum number of nodes in a segment executed in bulk (excluding
variables) in fwd pass.
+ size_t segment_num_nodes_threshold_fwd =
Imperative::BulkExecMaxNodeTrainFwd();
+ // The maximum number of nodes in a segment executed in bulk (excluding
variables) in bwd pass.
+ size_t segment_num_nodes_threshold_bwd =
Imperative::BulkExecMaxNodeTrainBwd();
// create forward segments for training
size_t topo_start = 0;
+ size_t segment_node_count = 0;
for (size_t nid = 0; nid < num_forward_nodes_; nid++) {
Review comment:
Should be fine to always use
```
bool ignore_node = node->is_variable() || op_node.skip_exec_node ||
op_node.exec == nullptr;
```
`skip_exec_node` defaults to False. Is it set to True if the node should be
skipped during execution. This is usually for elemwise_add op during backward
pass. It is set when the elemwise_add op is skipped, and the req for the
corresponding entry at the previous op is set to `kAddTo`. For forward pass,
`skip_exec_node` should always be False.
https://github.com/apache/incubator-mxnet/blob/992c3c0dd90c0723de6934e826a49bad6569eeac/src/executor/exec_pass.h#L162
It should also be ok to check `op_node.exec` for forward ops, as non
variable nodes are assigned with exec:
https://github.com/apache/incubator-mxnet/blob/master/src/executor/graph_executor.cc#L1089-L1094
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
