On 10/20/20 8:19 AM, PatricZhao wrote:
It will be more convenient for setting FAZ to true by default. The only concern
is that if it affects the training accuracy (suppose very limited).
We have encountered several performance issues with denormal computation in the
past but only happen in the user's debugging mode by randomly generated
numbers. Thus, I am not sure if this issue will be happening in real cases.
Let's wait for a while for the inputs from other members :)
This will also depend on application and floating point implementation
on device being used. May be helpful to have some idea on effects on
speed and accuracy - probably algorithm being used for training will be
important as well. Is there any useful empirical or theoretical research
in this area?
