TristonC commented on a change in pull request #20262:
URL: https://github.com/apache/incubator-mxnet/pull/20262#discussion_r654715291
##########
File path: benchmark/python/control_flow/rnn.py
##########
@@ -107,11 +107,11 @@ def run_benchmark(cell_type, ctx, seq_len, batch_size,
hidden_dim):
res = layer(inputs, states)
if is_train:
res.backward()
- mx.nd.waitall()
+ mx.npx.waitall()
tock = time()
times.append((tock - tick) * 1000.0)
times = times[args.warmup_rounds: ]
- print("Time used: mean = %.3f ms, std = %.3f ms" % (np.mean(times),
np.std(times)))
+ print("Time used: mean = %.3f ms, std = %.3f ms" % (onp.mean(times),
onp.std(times)))
Review comment:
Will mxnet np provide the mean and std function?
##########
File path: benchmark/python/control_flow/rnn.py
##########
@@ -24,8 +24,8 @@
from time import time
import mxnet as mx
-import numpy as np
-from mxnet import gluon
+import numpy as onp
Review comment:
Is the 'o' in onp as original as this is original numpy? It will be
confusing as np being well known as numpy for short.
##########
File path: include/mxnet/c_api.h
##########
@@ -1399,6 +1399,14 @@ MXNET_DLL int
MXNDArrayGetDeferredComputeSymbol(NDArrayHandle *output_handles,
int num_outputs,
SymbolHandle *out);
+/*!
+ * \brief Clear the info node associated with the arrays.
+ * \param arrays array handles of arrays
+ * \param num number of arrays
+ * \return 0 when success, -1 when failure happens
Review comment:
-1 otherwise
##########
File path: include/mxnet/c_api.h
##########
@@ -1399,6 +1399,14 @@ MXNET_DLL int
MXNDArrayGetDeferredComputeSymbol(NDArrayHandle *output_handles,
int num_outputs,
SymbolHandle *out);
+/*!
+ * \brief Clear the info node associated with the arrays.
Review comment:
The brief is not obvious with the function name. It is more about how
the deferred compute is handled.
##########
File path:
docs/python_docs/python/tutorials/packages/gluon/blocks/custom-layer.md
##########
@@ -131,50 +128,45 @@ Output:
[-0.05046433]
[-1.2375476 ]
[-0.15506986]]
-<NDArray 5x1 @cpu(0)>
```
## Parameters of a custom layer
-Usually, a layer has a set of associated parameters, sometimes also referred
as weights. This is an internal state of a layer. Most often, these parameters
are the ones, that we want to learn during backpropogation step, but sometimes
these parameters might be just constants we want to use during forward pass.
-
-All parameters of a block are stored and accessed via
[ParameterDict](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/parameter.py#L508)
class. This class helps with initialization, updating, saving and loading of
the parameters. Each layer can have multiple set of parameters, and all of them
can be stored in a single instance of the `ParameterDict` class. On a block
level, the instance of the `ParameterDict` class is accessible via
`self.params` field, and outside of a block one can access all parameters of
the network via
[collect_params()](https://mxnet.apache.org/api/python/gluon/gluon.html#mxnet.gluon.Block.collect_params)
method called on a `container`. `ParameterDict` uses
[Parameter](https://mxnet.apache.org/api/python/gluon/gluon.html#mxnet.gluon.Parameter)
class to represent parameters inside of Apache MxNet neural network. If
parameter doesn't exist, trying to get a parameter via `self.params` will
create it automatically.
+Usually, a layer has a set of associated parameters, sometimes also referred
as weights. This is an internal state of a layer. Most often, these parameters
are the ones, that we want to learn during backpropogation step, but sometimes
these parameters might be just constants we want to use during forward pass.
The parameters are usually represented as
[Parameter](https://mxnet.apache.org/api/python/gluon/gluon.html#mxnet.gluon.Parameter)
class inside of Apache MxNet neural network.
Review comment:
MxNet -> MXNet
##########
File path:
docs/python_docs/python/tutorials/packages/gluon/blocks/custom-layer.md
##########
@@ -56,41 +56,38 @@ The rest of methods of the `Block` class are already
implemented, and majority o
Looking into implementation of [existing
layers](https://mxnet.apache.org/api/python/gluon/nn.html), one may find that
more often a block inherits from a
[HybridBlock](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/block.py#L428),
instead of directly inheriting from `Block`.
-The reason for that is that `HybridBlock` allows to write custom layers that
can be used in imperative programming as well as in symbolic programming. It is
convinient to support both ways, because the imperative programming eases the
debugging of the code and the symbolic one provides faster execution speed. You
can learn more about the difference between symbolic vs. imperative programming
from [this article](https://mxnet.apache.org/api/architecture/overview.html).
+The reason for that is that `HybridBlock` allows to write custom layers in
imperative programming style, while computing in a symbolic way. It unifies the
flexibility of imperative programming with the performance benefits of symbolic
programming. You can learn more about the difference between symbolic vs.
imperative programming from [this
article](https://mxnet.apache.org/api/architecture/overview.html).
Review comment:
between ... and for " the difference between symbolic vs. imperative
programming"
##########
File path:
docs/python_docs/python/tutorials/packages/gluon/blocks/custom-layer.md
##########
@@ -185,23 +177,23 @@ def print_params(title, net):
Helper function to print out the state of parameters of
NormalizationHybridLayer
"""
print(title)
- hybridlayer_params = {k: v for k, v in net.collect_params().items() if
'normalizationhybridlayer' in k }
+ hybridlayer_params = {k: v for k, v in net.collect_params().items()}
for key, value in hybridlayer_params.items():
print('{} = {}\n'.format(key, value.data()))
net = gluon.nn.HybridSequential() # Define a
Neural Network as a sequence of hybrid blocks
net.add(Dense(5)) # Add Dense layer
with 5 neurons
net.add(NormalizationHybridLayer(hidden_units=5,
- scales = nd.array([2]))) # Add our custom
layer
+ scales = np.array([2]))) # Add our custom
layer
Review comment:
What about just say # Add a customer layer
##########
File path: python/mxnet/gluon/block.py
##########
@@ -1039,7 +1038,9 @@ def forward(self, x):
"""
def __init__(self):
super(HybridBlock, self).__init__()
- self._v2 = inspect.unwrap(self.hybrid_forward.__func__) is
HybridBlock.hybrid_forward
+ assert hasattr(self, "hybrid_forward") is False, (
+ "Starting from MXNet2.0, Gluon2.0 with forward interface will be
used instead of "
Review comment:
Does both MXNet2.0 and Gluon2.0 need to be met at the same time?
Propose:
'forward' instead of 'hybrid_forward' interfaces needs to be used starting
from Gluon 2.0. ......
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
