ehsanmok commented on code in PR #13074:
URL: https://github.com/apache/tvm/pull/13074#discussion_r996186639
##########
python/tvm/relay/frontend/onnx.py:
##########
@@ -944,6 +946,36 @@ def _impl_v1(cls, inputs, attr, params):
return Gelu._impl_v1([inp], attr, params)
+class LayerNormalization(OnnxOpConverter):
+ """Operator converter for LayerNormalization from Microsoft onnxruntime
contrib opset."""
+
+ @classmethod
+ def _impl_v17(cls, inputs, attr, params):
+ x = inputs[0]
+ gamma = inputs[1]
+ beta = inputs[2]
+ axis = attr.get("axis", -1)
+ eps = attr.get("epsilon", 1e-5)
+ # according to the onnx doc, given the int axis (default -1)
+ # to compute the mean and inv_stdev which are of dim [d[0], ...,
d[axis-1], 1, ..., 1]
+ # the actual computation is over (axis, ..., rank(x) - 1) axes
+ # see
https://github.com/onnx/onnx/blob/main/docs/Changelog.md#layernormalization-17
+ rank = len(infer_shape(x))
+ axis = tuple(range(axis, rank)) if axis >= 0 else tuple(range(rank +
axis, rank))
+ dtype = infer_type(x).checked_type.dtype
+ mean = _op.mean(x, axis, keepdims=True)
Review Comment:
Isn't that (the point of rely) gets optimized over the simplification passes
that rely does so we don't need to worry too much about it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
