slitsey edited a comment on issue #11466: [MXNET-560] Add temperature parameter in Softmax operator URL: https://github.com/apache/incubator-mxnet/pull/11466#issuecomment-403158593 @haojin2 That makes sense. To play devil's advocate, I would argue that having six different variations of softmax is a bit ridiculous, and more confusing than adding a single parameter to existing functions. Since the default behavior is unchanged, most users won't even know the difference from a UX perspective. You can't create a new function every time someone wants to add a new feature to an existing function; that kind of function proliferation generates far more user confusion than adding a parameter they can safely ignore. The performance regression argument is much more significant. However, while I'm not an expert on this, I suspect that adding an if statement to a function's execution is about as minimal a performance drop as you can get. (Especially given that's its just deciding whether or not to perform a floating point division operation!) Most other modifications would have a more substantial impact. Feel free to correct me if I'm mistaken on this point. Finally, making an enhancement to an important function like softmax can itself induce use of that feature. I wouldn't be surprised if the fraction of softmax users that employ this feature ends up being higher than one may think, given that it's a hyperparameter applicable to most use cases that tends to be ignored only due to ignorance or to maintain simplicity. Anyway, I'll let everyone else weigh in. For my own use case, having new SoftmaxWithTemperature functions would be fine, but I suspect for the library as a whole, it may be better to incorporate the change within the existing functions.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
