eric-haibin-lin commented on a change in pull request #14830: [WIP] Use env var
to enforce safe accumulation in ReduceAxesCompute
URL: https://github.com/apache/incubator-mxnet/pull/14830#discussion_r280998687
##########
File path: src/operator/tensor/broadcast_reduce_op.h
##########
@@ -1183,12 +1183,23 @@ void LpNormCompute(const nnvm::NodeAttrs& attrs,
} else {
small = ReduceAxesShapeImpl(inputs[0].shape_, param.axis, true, false);
}
+
if (param.ord == 1) {
- ReduceAxesComputeImpl<xpu, mshadow_op::sum, true, false, mshadow_op::abs>(
+ if (dmlc::GetEnv("MXNET_ENFORCE_SAFE_ACCUMULATION", false)) {
+ ReduceAxesComputeImpl<xpu, mshadow_op::sum, true, false,
mshadow_op::abs>(
+ ctx, inputs, req, outputs, small);
+ } else {
+ ReduceAxesComputeImpl<xpu, mshadow_op::sum, false, false,
mshadow_op::abs>(
ctx, inputs, req, outputs, small);
+ }
} else if (param.ord == 2) {
- ReduceAxesComputeImpl<xpu, mshadow_op::nrm2, true, false,
mshadow_op::identity>(
+ if (dmlc::GetEnv("MXNET_ENFORCE_SAFE_ACCUMULATION", false)) {
Review comment:
@anirudh2290 that may not always be true. For example, for the log_softmax
operator with fp16 inputs, you could just have fp32 acc_type and fp32 out_type.
Fp16 output does not make sense here because it always overflows. I don't see
an assumption that "when out_dtype is provided safe_accumulation is then set to
x"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
