https://issues.apache.org/bugzilla/show_bug.cgi?id=43940
--- Comment #13 from Vincent Hennebert <vhenneb...@gmail.com> 2011-12-20 15:00:14 UTC --- While looking at the patch something struck me as odd in the requirements. Taking the same example of decimals being 4 and precision 8, 1.00001 will be rounded to 1, while 0.99999999 will be kept as is. Yet the latter is much closer to 1 than the former (1000 times, to be precise). So if rounding had to be done, that would rather have to be the other way round, that is: keep 1.00001 as is and round 0.99999999 up to 1. It seems to me that if drawing commands are used with numbers around 1, then the result is likely to look sub-optimal. Especially since transformation matrices can potentially scale up rounding errors quite a bit. At the other end of the spectrum, 12345678901.2345 will be output as is (15 significant digits), while 1.23456789012345 will be rounded to 1.2346 (5 significant digits). What justifies to have 15 significant digits on one hand, and only 5 on the other hand? I understand that this exercise has to do with outputting real numbers to PDF or PostScript (or other formats). According to the PDF Reference, Acrobat uses single-precision floating-point numbers to represent reals (the 'float' type in Java). Given that a float has approximately 8 significant digits —which is actually what the PostScript Reference puts in its "Implementation Limits" section— I suggest that we use this as the sole parameter. Now, would we really want this parameter to be configurable? Using a smaller number of significant digits probably makes no sense; using a higher number may allow to achieve higher precision with those applications (viewers or printers) that go beyond Acrobat and use doubles to represent real numbers. Such applications may actually become more and more common with the rise of 64-bit processors. Then again, will that increased precision make a significant difference in the context of drawing graphics? Any thoughts on this are welcome. As a side note, if we choose to fix the precision to 8 then the implementation becomes almost trivial: just convert the double into a float and 'de-scientificize' the output of toString. Thanks, Vincent -- Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug.
