On 1 Nov 2011, at 3:24pm, Ryan Belcher wrote: > I do have one other question. Are there more downsides to using a tolerance > value in my comparisons? It seems to me that either going the integer or > tolerance route, I'm committing to a fixed precision for base and thick. If > I know that base and thick never use more than .0001 of precision and sqlite > uses 13 digits then I should be safe using .0000001 as a tolerance value, > right?
No. Because it's impossible to express the number .0000001 in binary. Whatever numbers you hold in floating point, you should always be prepared to lose some precision any time you do any maths, including comparisons. I'd recommend that you find out what your measurement precision is and use that as integers, or in your training explain to your users that they should always expect the last shown digit to be a little fuzzy. Simon. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
