On Thu, Mar 24, 2011 at 8:46 AM, Miklos Szigetvari <[email protected]> wrote: > Hi > > We have always ignored this small rounding differences, but here > the "end effect" was that > the generated document was different, as "hyphenation" , word > splitting worked differently.
You must have a huge canvas that you're using floating point to position characters in a line. Missing 1 pixel on 8" with 4800 dpi would be an error of 1e-6. Think I can do a lot of computation even in 4-byte floats and not get close to that. Sound more like some programming that tricked the compiler to fold an intermediate to integer or so... Maybe even optimizer level? We had something where a 64-bit integer was M = N * 2**20 -1 and we wanted (M+1) in a float F. Doing the float(M)+1 does not work well for large N while float(M+1) can be done very accurate... Rob - wondering whether rounding errors also get my Sudoku puzzles wrong :-)
