I'm aware but I think consistent results are more important than speed. BQN's approach is to provide •math.Sum for fast sums, which is a little over 10x faster for floats on AVX2 as you say.
Marshall On Mon, Jan 09, 2023 at 09:26:12PM -0500, Henry Rich wrote: > It's impossible NOT to 'rearrange' +/, if you want speed. With 1 ALU, you > add 0+1+2... except: suppose your add latency is 4? Then you have 4 > accumulators and interleave the addition of 0+4+8+12, 1+5+9+13, etc. > > But now AVX comes along and you add 4 at a time, so with 4 accumulators you > are adding 0+16+32+... etc and then there are 6 ways to combine the > accumulators. > > With AVX512 everything changes again. > > Henry Rich > > On 1/9/2023 8:32 PM, Marshall Lochbaum wrote: > > Well, true, I'm not in favor of rearranging +/ either. The dangers of > > floating point don't include nondeterminism, unless you make them. > > > > However, I also think matrix products have it worse. Numbers with widely > > varying exponents are a bit of an edge case. But when you're multiplying > > a few large matrices together they can show up naturally, so I expect > > it's not so rare to have a product that's numerically stable in one > > direction and not in the other. > > > > Marshall > > > > On Mon, Jan 09, 2023 at 05:52:34PM -0600, Omar Antolín Camarena wrote: > > > But that's just normal floating non-associativity. It happens even for > > > addition of "integers": > > > > > > 1 + (_1e19 + 1e19) > > > 1 > > > (1 + _1e19) + 1e19 > > > 0 > > > > > > People using floating point are probably aware of the dangers or at least > > > should be. > > > > > > -- > > > Omar > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
