Re: [julia-users] Fused-Multiply-Add function?
I've opened an issue (https://github.com/JuliaLang/julia/issues/17593) - and realized that with Julia v0.5 we may not need this at all. :-) On Saturday, July 23, 2016 at 8:47:57 PM UTC+2, Stefan Karpinski wrote: > > I would file an issue about having and fma! function. Since it's only > sensible for matrices (as far as I can tell), there doesn't seem to be a > corresponding need for muladd! since the fusion is at the collection level. > > On Sat, Jul 23, 2016 at 2:31 PM, Oliver Schulz > wrote: > >> Oh, sure - I was actually mainly thinking about matrices. I was looking >> for something like fma! in ArrayFire (without success), and then wondered >> what it might be called in Base. >> >> On Saturday, July 23, 2016 at 8:04:31 PM UTC+2, Stefan Karpinski wrote: >>> >>> They don't make sense for scalars but they could be added for matrices. >>> >>> On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz >> > wrote: >>> Hi Stefan, sorry, yes, I had somehow overlooked fma. Mainly I was looking for an in-place version though, like fma! and muladd!. Is there a reason those don't exist? On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski wrote: > > Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there > are both fma and muladd operations with different purposes: > > help?> fma > > > > search: fma findmax @fastmath UniformScaling > > > > > > > > fma(x, y, z) > > > > > > > > Computes x*y+z without rounding the intermediate result x*y. On some > systems this is significantly more expensive than x*y+z. fma is used to > improve accuracy in certain algorithms. See muladd. > > > > > > > > > help?> muladd > > > > search: muladd > > > > > > > > muladd(x, y, z) > > > > > > > > Combined multiply-add, computes x*y+z in an efficient manner. This > may on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is > used to improve performance. See fma. > > On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz < > oliver...@tu-dortmund.de> wrote: > >> Does Julia have a standardized FMA (Fused-Multiply-Add) function? >> Like fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU >> libraries, etc. can provide optimized versions? >> > > >>> >
Re: [julia-users] Fused-Multiply-Add function?
fma is a very specific operation defined by an IEEE standard. BLAS axpy does not implement it. If you extend the fma definition to linear algebra, then you'd arrive at a definition such as "no intermediate result will be rounded". The BLAS definition of axpy does not state that, and it's unlikely that any real-world implementation would do this. I'd thus that that axpy is equivalent to muladd, without any requirement on intermediate results, or the order in which things are evaluated. -erik On Sun, Jul 24, 2016 at 12:10 PM, Stefan Karpinski wrote: > Maybe it does. It's unclear to me whether operations like BLAS apxy > implement fma or muladd. > > On Sunday, July 24, 2016, Oliver Schulz > wrote: > >> Uh, sorry, I don't quite get that. I thought muladd was basically that >> same as fma - with the difference that fma has to do it without rounding >> the intermediate result, while muladd is free to do whatever is most >> efficient on the given hardware? But why wouldn't make muladd! make sense >> for collections then? >> >> On Saturday, July 23, 2016 at 8:47:57 PM UTC+2, Stefan Karpinski wrote: >>> >>> I would file an issue about having and fma! function. Since it's only >>> sensible for matrices (as far as I can tell), there doesn't seem to be a >>> corresponding need for muladd! since the fusion is at the collection level. >>> >>> On Sat, Jul 23, 2016 at 2:31 PM, Oliver Schulz >> > wrote: >>> Oh, sure - I was actually mainly thinking about matrices. I was looking for something like fma! in ArrayFire (without success), and then wondered what it might be called in Base. On Saturday, July 23, 2016 at 8:04:31 PM UTC+2, Stefan Karpinski wrote: > > They don't make sense for scalars but they could be added for matrices. > > On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz < > oliver...@tu-dortmund.de> wrote: > >> Hi Stefan, >> >> sorry, yes, I had somehow overlooked fma. Mainly I was looking for an >> in-place version though, like fma! and muladd!. Is there a reason those >> don't exist? >> >> On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski >> wrote: >>> >>> Yes: https://github.com/JuliaLang/julia/issues/6330. In short, >>> there are both fma and muladd operations with different purposes: >>> >>> help?> fma >>> >>> >>> search: fma findmax @fastmath UniformScaling >>> >>> >>> >>> >>> >>> fma(x, y, z) >>> >>> >>> >>> >>> >>> Computes x*y+z without rounding the intermediate result x*y. On >>> some systems this is significantly more expensive than x*y+z. fma is >>> used >>> to improve accuracy in certain algorithms. See muladd. >>> >>> >>> >>> >>> >>> help?> muladd >>> >>> >>> search: muladd >>> >>> >>> >>> >>> >>> muladd(x, y, z) >>> >>> >>> >>> >>> >>> Combined multiply-add, computes x*y+z in an efficient manner. This >>> may on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is >>> used to improve performance. See fma. >>> >>> On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz < >>> oliver...@tu-dortmund.de> wrote: >>> Does Julia have a standardized FMA (Fused-Multiply-Add) function? Like fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU libraries, etc. can provide optimized versions? >>> >>> > >>> -- Erik Schnetter http://www.perimeterinstitute.ca/personal/eschnetter/
Re: [julia-users] Fused-Multiply-Add function?
Maybe it does. It's unclear to me whether operations like BLAS apxy implement fma or muladd. On Sunday, July 24, 2016, Oliver Schulz wrote: > Uh, sorry, I don't quite get that. I thought muladd was basically that > same as fma - with the difference that fma has to do it without rounding > the intermediate result, while muladd is free to do whatever is most > efficient on the given hardware? But why wouldn't make muladd! make sense > for collections then? > > On Saturday, July 23, 2016 at 8:47:57 PM UTC+2, Stefan Karpinski wrote: >> >> I would file an issue about having and fma! function. Since it's only >> sensible for matrices (as far as I can tell), there doesn't seem to be a >> corresponding need for muladd! since the fusion is at the collection level. >> >> On Sat, Jul 23, 2016 at 2:31 PM, Oliver Schulz >> wrote: >> >>> Oh, sure - I was actually mainly thinking about matrices. I was looking >>> for something like fma! in ArrayFire (without success), and then wondered >>> what it might be called in Base. >>> >>> On Saturday, July 23, 2016 at 8:04:31 PM UTC+2, Stefan Karpinski wrote: They don't make sense for scalars but they could be added for matrices. On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz < oliver...@tu-dortmund.de> wrote: > Hi Stefan, > > sorry, yes, I had somehow overlooked fma. Mainly I was looking for an > in-place version though, like fma! and muladd!. Is there a reason those > don't exist? > > On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski wrote: >> >> Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there >> are both fma and muladd operations with different purposes: >> >> help?> fma >> >> >> search: fma findmax @fastmath UniformScaling >> >> >> >> >> >> fma(x, y, z) >> >> >> >> >> >> Computes x*y+z without rounding the intermediate result x*y. On >> some systems this is significantly more expensive than x*y+z. fma is used >> to improve accuracy in certain algorithms. See muladd. >> >> >> >> >> >> help?> muladd >> >> >> search: muladd >> >> >> >> >> >> muladd(x, y, z) >> >> >> >> >> >> Combined multiply-add, computes x*y+z in an efficient manner. This >> may on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is >> used to improve performance. See fma. >> >> On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz < >> oliver...@tu-dortmund.de> wrote: >> >>> Does Julia have a standardized FMA (Fused-Multiply-Add) function? >>> Like fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU >>> libraries, etc. can provide optimized versions? >>> >> >> >>
Re: [julia-users] Fused-Multiply-Add function?
Uh, sorry, I don't quite get that. I thought muladd was basically that same as fma - with the difference that fma has to do it without rounding the intermediate result, while muladd is free to do whatever is most efficient on the given hardware? But why wouldn't make muladd! make sense for collections then? On Saturday, July 23, 2016 at 8:47:57 PM UTC+2, Stefan Karpinski wrote: > > I would file an issue about having and fma! function. Since it's only > sensible for matrices (as far as I can tell), there doesn't seem to be a > corresponding need for muladd! since the fusion is at the collection level. > > On Sat, Jul 23, 2016 at 2:31 PM, Oliver Schulz > wrote: > >> Oh, sure - I was actually mainly thinking about matrices. I was looking >> for something like fma! in ArrayFire (without success), and then wondered >> what it might be called in Base. >> >> On Saturday, July 23, 2016 at 8:04:31 PM UTC+2, Stefan Karpinski wrote: >>> >>> They don't make sense for scalars but they could be added for matrices. >>> >>> On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz >> > wrote: >>> Hi Stefan, sorry, yes, I had somehow overlooked fma. Mainly I was looking for an in-place version though, like fma! and muladd!. Is there a reason those don't exist? On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski wrote: > > Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there > are both fma and muladd operations with different purposes: > > help?> fma > > > > search: fma findmax @fastmath UniformScaling > > > > > > > > fma(x, y, z) > > > > > > > > Computes x*y+z without rounding the intermediate result x*y. On some > systems this is significantly more expensive than x*y+z. fma is used to > improve accuracy in certain algorithms. See muladd. > > > > > > > > > help?> muladd > > > > search: muladd > > > > > > > > muladd(x, y, z) > > > > > > > > Combined multiply-add, computes x*y+z in an efficient manner. This > may on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is > used to improve performance. See fma. > > On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz < > oliver...@tu-dortmund.de> wrote: > >> Does Julia have a standardized FMA (Fused-Multiply-Add) function? >> Like fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU >> libraries, etc. can provide optimized versions? >> > > >>> >
Re: [julia-users] Fused-Multiply-Add function?
Yes, I know axpy. However - as far a I understand - that is explicitly a BLAS function. So it's not something that, e.g., a GPU library would provide specialized methods for. Also, axpy always results in a call, even for small arrays (please correct me if I'm wrong). I think in-place multiply-add for collections is something so basic and general that it would to well in Base. On Sunday, July 24, 2016 at 12:24:02 AM UTC+2, Jutho wrote: > > Did you check axpy! which originates from the BLAS tradition?
Re: [julia-users] Fused-Multiply-Add function?
Did you check axpy! which originates from the BLAS tradition?
Re: [julia-users] Fused-Multiply-Add function?
I would file an issue about having and fma! function. Since it's only sensible for matrices (as far as I can tell), there doesn't seem to be a corresponding need for muladd! since the fusion is at the collection level. On Sat, Jul 23, 2016 at 2:31 PM, Oliver Schulz wrote: > Oh, sure - I was actually mainly thinking about matrices. I was looking > for something like fma! in ArrayFire (without success), and then wondered > what it might be called in Base. > > On Saturday, July 23, 2016 at 8:04:31 PM UTC+2, Stefan Karpinski wrote: >> >> They don't make sense for scalars but they could be added for matrices. >> >> On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz >> wrote: >> >>> Hi Stefan, >>> >>> sorry, yes, I had somehow overlooked fma. Mainly I was looking for an >>> in-place version though, like fma! and muladd!. Is there a reason those >>> don't exist? >>> >>> On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski wrote: Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there are both fma and muladd operations with different purposes: help?> fma search: fma findmax @fastmath UniformScaling fma(x, y, z) Computes x*y+z without rounding the intermediate result x*y. On some systems this is significantly more expensive than x*y+z. fma is used to improve accuracy in certain algorithms. See muladd. help?> muladd search: muladd muladd(x, y, z) Combined multiply-add, computes x*y+z in an efficient manner. This may on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is used to improve performance. See fma. On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz < oliver...@tu-dortmund.de> wrote: > Does Julia have a standardized FMA (Fused-Multiply-Add) function? Like > fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU > libraries, > etc. can provide optimized versions? > >>
Re: [julia-users] Fused-Multiply-Add function?
Oh, sure - I was actually mainly thinking about matrices. I was looking for something like fma! in ArrayFire (without success), and then wondered what it might be called in Base. On Saturday, July 23, 2016 at 8:04:31 PM UTC+2, Stefan Karpinski wrote: > > They don't make sense for scalars but they could be added for matrices. > > On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz > wrote: > >> Hi Stefan, >> >> sorry, yes, I had somehow overlooked fma. Mainly I was looking for an >> in-place version though, like fma! and muladd!. Is there a reason those >> don't exist? >> >> On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski wrote: >>> >>> Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there >>> are both fma and muladd operations with different purposes: >>> >>> help?> fma >>> >>> >>> search: fma findmax @fastmath UniformScaling >>> >>> >>> >>> >>> >>> fma(x, y, z) >>> >>> >>> >>> >>> >>> Computes x*y+z without rounding the intermediate result x*y. On some >>> systems this is significantly more expensive than x*y+z. fma is used to >>> improve accuracy in certain algorithms. See muladd. >>> >>> >>> >>> >>> >>> help?> muladd >>> >>> >>> search: muladd >>> >>> >>> >>> >>> >>> muladd(x, y, z) >>> >>> >>> >>> >>> >>> Combined multiply-add, computes x*y+z in an efficient manner. This may >>> on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is used to >>> improve performance. See fma. >>> >>> On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz >> > wrote: >>> Does Julia have a standardized FMA (Fused-Multiply-Add) function? Like fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU libraries, etc. can provide optimized versions? >>> >>> >
Re: [julia-users] Fused-Multiply-Add function?
They don't make sense for scalars but they could be added for matrices. On Sat, Jul 23, 2016 at 1:54 PM, Oliver Schulz wrote: > Hi Stefan, > > sorry, yes, I had somehow overlooked fma. Mainly I was looking for an > in-place version though, like fma! and muladd!. Is there a reason those > don't exist? > > On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski wrote: >> >> Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there are >> both fma and muladd operations with different purposes: >> >> help?> fma >> >> >> search: fma findmax @fastmath UniformScaling >> >> >> >> >> >> fma(x, y, z) >> >> >> >> >> >> Computes x*y+z without rounding the intermediate result x*y. On some >> systems this is significantly more expensive than x*y+z. fma is used to >> improve accuracy in certain algorithms. See muladd. >> >> >> >> >> >> help?> muladd >> >> >> search: muladd >> >> >> >> >> >> muladd(x, y, z) >> >> >> >> >> >> Combined multiply-add, computes x*y+z in an efficient manner. This may >> on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is used to >> improve performance. See fma. >> >> On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz >> wrote: >> >>> Does Julia have a standardized FMA (Fused-Multiply-Add) function? Like >>> fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU libraries, >>> etc. can provide optimized versions? >>> >> >>
Re: [julia-users] Fused-Multiply-Add function?
Hi Stefan, sorry, yes, I had somehow overlooked fma. Mainly I was looking for an in-place version though, like fma! and muladd!. Is there a reason those don't exist? On Saturday, July 23, 2016 at 7:50:30 PM UTC+2, Stefan Karpinski wrote: > > Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there are > both fma and muladd operations with different purposes: > > help?> fma > > > search: fma findmax @fastmath UniformScaling > > > > > > fma(x, y, z) > > > > > > Computes x*y+z without rounding the intermediate result x*y. On some > systems this is significantly more expensive than x*y+z. fma is used to > improve accuracy in certain algorithms. See muladd. > > > > > > help?> muladd > > > search: muladd > > > > > > muladd(x, y, z) > > > > > > Combined multiply-add, computes x*y+z in an efficient manner. This may > on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is used to > improve performance. See fma. > > On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz > wrote: > >> Does Julia have a standardized FMA (Fused-Multiply-Add) function? Like >> fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU libraries, >> etc. can provide optimized versions? >> > >
Re: [julia-users] Fused-Multiply-Add function?
Yes: https://github.com/JuliaLang/julia/issues/6330. In short, there are both fma and muladd operations with different purposes: help?> fma search: fma findmax @fastmath UniformScaling fma(x, y, z) Computes x*y+z without rounding the intermediate result x*y. On some systems this is significantly more expensive than x*y+z. fma is used to improve accuracy in certain algorithms. See muladd. help?> muladd search: muladd muladd(x, y, z) Combined multiply-add, computes x*y+z in an efficient manner. This may on some systems be equivalent to x*y+z, or to fma(x,y,z). muladd is used to improve performance. See fma. On Sat, Jul 23, 2016 at 1:40 PM, Oliver Schulz wrote: > Does Julia have a standardized FMA (Fused-Multiply-Add) function? Like > fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU libraries, > etc. can provide optimized versions? >
[julia-users] Fused-Multiply-Add function?
Does Julia have a standardized FMA (Fused-Multiply-Add) function? Like fma(A, B, factor) and fma!(dest, A, B, factor), so that that GPU libraries, etc. can provide optimized versions?
