wzh99 commented on issue #11696: URL: https://github.com/apache/tvm/issues/11696#issuecomment-1154691081
@ganler Thanks for your reply. Your explanation is reasonable. I agree that fast math implementation does not always need to consider all kinds of invalid inputs. Actually I does not intentionally feed `nan` to `tanh` to discover this inconsistency. Instead, I generated a computation graph and feed it with valid inputs. The `nan`s appear in intermediate results. The test case shown above is a reduced version of what is generated by my fuzzer. I think that the `nan`s in intermediate values are not completely avoidable for arbitrary tensor programs. Just now I figure out a quick fix of this inconsistency without introducing overhead. I replace the clamping code in [tvm/include/tvm/topi/elemwise.h](https://github.com/apache/tvm/blob/8341e33d05868b7bb8496c913679b7951836f3b9/include/tvm/topi/elemwise.h) with the following code: ```c++ auto x = maximum(make_const(in->dtype, -9.0), minimum(make_const(in->dtype, 9.0), in)); ``` And the result at optimization level 4 is consistent with lower levels: ``` opt_level=0 [nan] opt_level=1 [nan] opt_level=2 [nan] opt_level=3 [nan] opt_level=4 [nan] ``` I just exchange the two arguments in `maximum` and `minimum`. The reason why this works is probably that `minimum(x, 9)` is implemented by `x <= 9 ? x : 9`. If `x` is `nan`, its comparison with any floating point is false. In this case `x <= 9 ? x : 9` evaluates to 9, which is a wrong clamp. In contrast, `9 <= x ? 9 : x` evaluates to `nan` and `x` is not clamped to a wrong value. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
