masahi commented on a change in pull request #5249: [PYTORCH]LayerNorm support
added
URL: https://github.com/apache/incubator-tvm/pull/5249#discussion_r404391408
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -503,6 +503,34 @@ def _impl(inputs, input_types):
scale=scale)
return _impl
+def _get_dims(data):
+ import torch
+ if isinstance(data, _expr.Expr):
+ dims = _infer_shape(data)
+ elif isinstance(data, list):
+ dims = data
+ elif isinstance(data, (torch.Tensor, np.ndarray)):
+ dims = data.shape
+ else:
+ msg = "Data type %s could not be parsed" % type(data)
+ raise AssertionError(msg)
+ return dims
+
+def _layer_norm():
+ def _impl(inputs, input_types):
+ data = inputs[0]
+ ndims = len(_get_dims(inputs[1]))
+ assert ndims == 1, "Support only normalization over last one
dimension."
+
+ return _op.nn.layer_norm(data,
+ gamma=inputs[1],
+ beta=inputs[2],
+ axis=-1,
Review comment:
it seems gamma is inputs[2] and beta is inputs[3]. @siju-samuel Can you
confirm?
I get an error using this new conversion in my test. Above change fixed it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
