AndrewZhaoLuo commented on PR #12935: URL: https://github.com/apache/tvm/pull/12935#issuecomment-1347200737
@masahi Ah yes, I will try to get to this during the next week where I will have time. 1. Figure out the behavior for autotvm and autoscheduler. 2. The topi kernel for fused layernorm is known to be numerically unstable. We should replace it with a more stable version e.g. https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Parallel_algorithm first. 3. https://github.com/apache/tvm/pull/13532 this PR is included here to support fp16 layernorm, but needs some refactoring to avoid casting cost. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
