abhinavs95 edited a comment on issue #14373: Passing parameters to HybridBlocks and not using them URL: https://github.com/apache/incubator-mxnet/issues/14373#issuecomment-479228739 @whamza15 This is not an issue of not using all variables in hybrid_forward as the following test works ``` import mxnet.gluon as gl import mxnet as mx class EmbeddingBlock(gl.HybridBlock): def __init__(self, num_toks, dim, **kwargs): super(EmbeddingBlock, self).__init__(**kwargs) self.emb = gl.nn.Embedding(num_toks, dim) def hybrid_forward(self, F, x, valid_length): # NOTE valid_length is not used return self.emb(x) net = EmbeddingBlock(10, 100) net.initialize() net.hybridize() x1 = mx.nd.array(range(8)).reshape(2,-1) vl1 = mx.nd.array([3,2]) x2 = mx.nd.array(range(8)).reshape(2,-1) vl2 = mx.nd.array([3,2]) net(x1, vl1) print(net.collect_params()) ``` EDIT: The above test works because deferred initialization is not used for embedding layers. For layers using deferred initialization like `nn.dense` the issue exists as can be verified using the following: ``` class Net(gl.HybridBlock): def __init__(self, **kwargs): super(Net, self).__init__(**kwargs) self.dense = gl.nn.Dense(3, flatten=False) def hybrid_forward(self, F, x, v1): return self.dense(x) net = Net() net.initialize() net.hybridize() x = mx.nd.array(range(8)).reshape(2,-1) v1 = mx.nd.array([3,2]) net(x, v1) ``` Error Message: ``` /anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py:540: UserWarning: The 1-th input to HybridBlock is not used by any computation. Is this intended? out = self.forward(*args) infer_shape error. Arguments: data0: (2, 4) data1: (2,) Traceback (most recent call last): File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 803, in _call_cached_op for is_arg, i in self._cached_op_args] File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 803, in <listcomp> for is_arg, i in self._cached_op_args] File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/parameter.py", line 494, in data return self._check_and_get(self._data, ctx) File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/parameter.py", line 208, in _check_and_get "num_features, etc., for network layers."%(self.name)) mxnet.gluon.parameter.DeferredInitializationError: Parameter 'dense0_weight' has not been initialized yet because initialization was deferred. Actual initialization happens during the first forward pass. Please pass one batch of data through the network before accessing Parameters. You can also avoid deferred initialization by specifying in_units, num_features, etc., for network layers. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 789, in _deferred_infer_shape self.infer_shape(*args) File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 862, in infer_shape self._infer_attrs('infer_shape', 'shape', *args) File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 851, in _infer_attrs **{i.name: getattr(j, attr) for i, j in zip(inputs, args)}) File "/anaconda3/lib/python3.7/site-packages/mxnet/symbol/symbol.py", line 996, in infer_shape res = self._infer_shape_impl(False, *args, **kwargs) File "/anaconda3/lib/python3.7/site-packages/mxnet/symbol/symbol.py", line 1126, in _infer_shape_impl ctypes.byref(complete))) File "/anaconda3/lib/python3.7/site-packages/mxnet/base.py", line 252, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [14:53:40] src/c_api/c_api_symbolic.cc:494: InferShapeKeyword argument name data1 not found. Candidate arguments: [0]data0 [1]dense0_weight [2]dense0_bias Stack trace returned 5 entries: [bt] (0) 0 libmxnet.so 0x000000011164e390 std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*> > >::destroy(std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, void*>*) + 2736 [bt] (1) 1 libmxnet.so 0x000000011164e13f std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*> > >::destroy(std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, void*>*) + 2143 [bt] (2) 2 libmxnet.so 0x0000000112c4a85e MXSymbolInferShape + 9582 [bt] (3) 3 libmxnet.so 0x0000000112c48b82 MXSymbolInferShape + 2194 [bt] (4) 4 libffi.6.dylib 0x000000010a0b1884 ffi_call_unix64 + 76 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "test_gl1.py", line 28, in <module> net(x, v1) File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 540, in __call__ out = self.forward(*args) File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 907, in forward return self._call_cached_op(x, *args) File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 805, in _call_cached_op self._deferred_infer_shape(*args) File "/anaconda3/lib/python3.7/site-packages/mxnet/gluon/block.py", line 793, in _deferred_infer_shape raise ValueError(error_msg) ValueError: Deferred initialization failed because shape cannot be inferred. [14:53:40] src/c_api/c_api_symbolic.cc:494: InferShapeKeyword argument name data1 not found. Candidate arguments: [0]data0 [1]dense0_weight [2]dense0_bias Stack trace returned 5 entries: [bt] (0) 0 libmxnet.so 0x000000011164e390 std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*> > >::destroy(std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, void*>*) + 2736 [bt] (1) 1 libmxnet.so 0x000000011164e13f std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*> > >::destroy(std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::NDArrayFunctionReg*>, void*>*) + 2143 [bt] (2) 2 libmxnet.so 0x0000000112c4a85e MXSymbolInferShape + 9582 [bt] (3) 3 libmxnet.so 0x0000000112c48b82 MXSymbolInferShape + 2194 [bt] (4) 4 libffi.6.dylib 0x000000010a0b1884 ffi_call_unix64 + 76 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
