siyiweigeHEW opened a new issue, #20184:
URL: https://github.com/apache/tvm/issues/20184
### Expected behavior
ONNX `Softplus` is defined (opset 1 through 22, no attributes) as:
```
Y = log(exp(X) + 1)
```
It must be evaluated for every finite input. There is **no `threshold`
parameter** in the ONNX specification. For `X = 20.1` in float64, `log(exp(X) +
1)` is exactly representable and should equal `20.10000000186501`.
### Actual behavior
The ONNX frontend hardcodes a `threshold` that the spec does not have, and
the lowering silently replaces `log(exp(x)+1)` with the linear function `x` for
`x > threshold`:
- `python/tvm/relax/frontend/onnx/onnx_frontend.py:1806-1807`:
```python
class Softplus(OnnxOpConverter):
@classmethod
def _impl_v1(cls, bb, inputs, attr, params):
dtype = inputs[0].struct_info.dtype
threshold = 10.0 if dtype == "float16" else 20.0
return relax.op.nn.softplus(inputs[0], threshold=threshold)
```
- `python/tvm/topi/nn/elemwise.py:94-96`:
```python
return tvm.tir.Select(
b * value > t, value, (1 / b) * tvm.tir.log(1 + tvm.tir.exp(b * value))
)
```
Consequence in **float64**: for `x` in roughly `(20, 30]`, TVM returns
exactly `x`, while the ONNX spec value `x + log(1 + exp(-x))` is fully
representable. `x = 20.1` gives an absolute error of `1.87e-9`:
| x (float64) | ONNX spec / `onnx.reference` | TVM | abs error |
|---|---|---|---|
| 20.0 | 20.000000002061153 | 20.000000002061153 | 0 (below threshold, exact
path) |
| 20.1 | 20.10000000186501 | **20.1** | **1.87e-9** |
| 25.0 | 25.000000000013888 | **25.0** | 1.39e-11 |
The threshold of `20` is appropriate for float32 (where `log(exp(x)+1)`
already rounds to `x` for `x ≳ 15` because the `+1` is lost), but it is far too
small for float64, where the exact value is representable up to `x ≈ 709`.
Note: in float32/float16 the linear fallback is numerically
indistinguishable from `log(exp(x)+1)` for ordinary inputs (max abs diff = 0 vs
onnxruntime and `onnx.reference` for `x ∈ [19, 50]`), and near overflow (`x >
~88` float32) onnxruntime *also* returns a finite value, so TVM agrees with
onnxruntime there. The clearly observable, spec-relevant deviation is the
float64 case above.
### Environment
- OS: Linux
- TVM: v0.24.dev0 (main branch, commit `262c6d2e0`, built 2026-02-11)
- Python: 3.11
- onnx: 1.20.1
- onnxruntime: 1.24.1 (CPU EP does not implement float64 `Softplus`;
`onnx.reference` used as the float64 oracle)
### Steps to reproduce
```python
"""Repro: ONNX Softplus float64 accuracy loss from the injected threshold."""
import numpy as np
import onnxruntime
from onnx import helper, TensorProto
from onnx.reference import ReferenceEvaluator
from tvm.relax.frontend.onnx import from_onnx
from tvm import relax
np.set_printoptions(precision=17)
def build(xs, dtype, opset=13):
dt = TensorProto.DOUBLE if dtype == "float64" else TensorProto.FLOAT
node = helper.make_node("Softplus", ["X"], ["Y"])
g = helper.make_graph(
[node], "g",
[helper.make_tensor_value_info("X", dt, list(xs.shape))],
[helper.make_tensor_value_info("Y", dt, None)], [])
m = helper.make_model(g, opset_imports=[helper.make_opsetid("", opset)])
m.ir_version = 8
return m
def run_tvm(m, shape, feeds):
mod = from_onnx(m, shape_dict={"X": list(shape)})
ex = relax.build(mod, target="llvm")
return relax.VirtualMachine(ex, tvm.cpu())["main"](feeds["X"]).numpy()
# float64: x = 20.1 is just above the hardcoded threshold
m = build(np.array([20.1]), "float64")
feeds = {"X": np.array([20.1], np.float64)}
print("onnx.reference:", ReferenceEvaluator(m).run(None, feeds)[0]) #
[20.10000000186501]
print("TVM :", run_tvm(m, [1], feeds)) #
[20.1]
# float32 near overflow (mechanism only; TVM agrees with onnxruntime)
m2 = build(np.array([100.0]), "float32")
f2 = {"X": np.array([100.0], np.float32)}
print("f32 onnxruntime:",
onnxruntime.InferenceSession(m2.SerializeToString()).run(None, f2)[0]) # [100.]
print("f32 onnx.reference:", ReferenceEvaluator(m2).run(None, f2)[0]) #
[inf]
print("f32 TVM :", run_tvm(m2, [1], f2)) #
[100.]
```
Actual output:
```
onnx.reference: [20.10000000186501]
TVM : [20.1]
f32 onnxruntime: [100.]
f32 onnx.reference: [inf]
f32 TVM : [100.]
```
### Possible fix
The ONNX `Softplus` has no threshold semantics. Options:
1. Do not pass a `threshold` for the ONNX converter (use a numerically
stable form without the hard linear fallback, e.g. compute `log(exp(x)+1)`
stably), or
2. Make the threshold dtype-aware: `20` is too small for float64 (should be
≥ ~37, where `log(1+exp(-x))` drops below float64 epsilon).
### Triage
* needs-triage
* bug
* relax
* frontend/onnx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
