On Mon, Nov 1, 2010 at 7:49 PM, Robert Kern <[email protected]> wrote:
> On Mon, Nov 1, 2010 at 20:21, Charles R Harris > <[email protected]> wrote: > > > > On Mon, Nov 1, 2010 at 5:30 PM, Joon <[email protected]> wrote: > >> > >> Hi, > >> > >> I just found that using dot instead of sum in numpy gives me better > >> results in terms of precision loss. For example, I optimized a function > with > >> scipy.optimize.fmin_bfgs. For the return value for the function, I tried > the > >> following two things: > >> > >> sum(Xb) - sum(denominator) > >> > >> and > >> > >> dot(ones(Xb.shape), Xb) - dot(ones(denominator.shape), denominator) > >> > >> Both of them are supposed to yield the same thing. But the first one > gave > >> me -589112.30492110562 and the second one gave me -589112.30492110678. > >> > >> In addition, with the routine using sum, the optimizer gave me "Warning: > >> Desired error not necessarily achieved due to precision loss." With the > >> routine with dot, the optimizer gave me "Optimization terminated > >> successfully." > >> > >> I checked the gradient value as well (I provided analytical gradient) > and > >> gradient was smaller in the dot case as well. (Of course, the the > magnitude > >> was e-5 to e-6, but still) > >> > >> I was wondering if this is well-known fact and I'm supposed to use dot > >> instead of sum whenever possible. > >> > >> It would be great if someone could let me know why this happens. > > > > Are you running on 32 bits or 64 bits? I ask because there are different > > floating point precisions on the 32 bit platform and the results can > depend > > on how the compiler does things. > > Eh, what? Are you talking about the sometimes-differing intermediate > precisions? I wasn't aware that was constrained to 32-bit processors. > > It seems to be more of a problem on 32 bits, what with the variety of sse*. OTOH, all 64 bit systems have at least sse2 available together with more sse registers and I believe the x87 instruction set is not available when running in 64 bit mode. I may be wrong about that, these nasty details are hard to verify, but that has also been my experience. Chuck
_______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
