fma is a very specific operation defined by an IEEE standard. BLAS axpy does not implement it.
If you extend the fma definition to linear algebra, then you'd arrive at a definition such as "no intermediate result will be rounded". The BLAS definition of axpy does not state that, and it's unlikely that any real-world implementation would do this. I'd thus that that axpy is equivalent to muladd, without any requirement on intermediate results, or the order in which things are evaluated. -erik On Sun, Jul 24, 2016 at 12:10 PM, Stefan Karpinski <[email protected]> wrote: > Maybe it does. It's unclear to me whether operations like BLAS apxy > implement fma or muladd. > > On Sunday, July 24, 2016, Oliver Schulz <[email protected]> > wrote: > >> Uh, sorry, I don't quite get that. I thought muladd was basically that >> same as fma - with the difference that fma has to do it without rounding >> the intermediate result, while muladd is free to do whatever is most >> efficient on the given hardware? But why wouldn't make muladd! make sense >> for collections then? >> >> On Saturday, July 23, 2016 at 8:47:57 PM UTC+2, Stefan Karpinski wrote: >>> >>> I would file an issue about having and fma! function. Since it's only >>> sensible for matrices (as far as I can tell), there doesn't seem to be a >>> corresponding need for muladd! since the fusion is at the collection level. >>> >>> On Sat, Jul 23, 2016 at 2:31 PM, Oliver Schulz <[email protected] >>> > wrote: >>> >>>> Oh, sure - I was actually mainly thinking about matrices. I was looking >>>> for something like fma! in ArrayFire (without success), and then wondered >>>> what it might be called in Base. >>>> >>>> On Saturday, July 23, 2016 at 8:04:31 PM UTC+2, Stefan Karpinski wrote: >>>>> >>>>> They don't make sense for scalars but they could be added for matrices. >>>>> >>>>> On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Stefan, >>>>>> >>>>>> sorry, yes, I had somehow overlooked fma. Mainly I was looking for an >>>>>> in-place version though, like fma! and muladd!. Is there a reason those >>>>>> don't exist? >>>>>> >>>>>> On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski >>>>>> wrote: >>>>>>> >>>>>>> Yes: https://github.com/JuliaLang/julia/issues/6330. In short, >>>>>>> there are both fma and muladd operations with different purposes: >>>>>>> >>>>>>> help?> fma >>>>>>> >>>>>>> >>>>>>> search: fma findmax @fastmath UniformScaling >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> fma(x, y, z) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Computes x*y+z without rounding the intermediate result x*y. On >>>>>>> some systems this is significantly more expensive than x*y+z. fma is >>>>>>> used >>>>>>> to improve accuracy in certain algorithms. See muladd. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> help?> muladd >>>>>>> >>>>>>> >>>>>>> search: muladd >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> muladd(x, y, z) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Combined multiply-add, computes x*y+z in an efficient manner. This >>>>>>> may on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is >>>>>>> used to improve performance. See fma. >>>>>>> >>>>>>> On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Does Julia have a standardized FMA (Fused-Multiply-Add) function? >>>>>>>> Like fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU >>>>>>>> libraries, etc. can provide optimized versions? >>>>>>>> >>>>>>> >>>>>>> >>>>> >>> -- Erik Schnetter <[email protected]> http://www.perimeterinstitute.ca/personal/eschnetter/
