I am trying to implement [task-specific weighting of multiple embeddings as in
Elmo](https://arxiv.org/pdf/1802.05365.pdf).
Currently, I initialized weights for multiple embeddings using
`self.param.get`. However, it throws me the error.
`AssertionError: Argument data must be Symbol instances, but got Parameter
elmoembedding0_weights (shape=(3,), dtype=<class 'numpy.float32'>)`.
I can call `x.data()` for non hybridized or `x.var()` for hybridized version
for the parameters. Is there a way to simply apply Softmax to parameters and
work with both versions? Thanks!
My code looks something like this
```
import mxnet as mx
import mxnet.gluon as gluon
class ElmoEmbedding(gluon.HybridBlock):
def __init__(self):
super(ElmoEmbedding, self).__init__()
with self.name_scope():
self.weights = self.params.get('weights',
shape=(3,),
init=mx.init.Constant(1.0))
self.scales = self.params.get('scales',
shape=(1,0),
init=mx.init.Constant(1.0))
def hybrid_forward(self, F, x, *args, **kwargs):
normalized_weights = F.softmax(self.weights)
weighted_x = F.dot(normalized_weights, x)
output = F.broadcast_mul(self.scales, weighted_x)
return output
net = ElmoEmbedding()
net.hybridize()
# create input
x = mx.ndarray.random.randn(3,100)
output = net(x)
print("output", output.shape)
```
---
[Visit
Topic](https://discuss.mxnet.apache.org/t/how-to-apply-f-softmax-to-gluon-parameters/6930/1)
or reply to this email to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.mxnet.apache.org/email/unsubscribe/97036f3ac6bc46f8d999282ce625d054f6fcc6ddb3d52b75e7e615d16c3d287d).
