ByteBuffer.formatDouble() is designed to be fast and output 5 significative digits, not 5 decimal digits in all cases. This compromise works well but I also realize that for the odd cases more precision may be needed. There's version in the CVS that will always output 6 decimal digits if ByteBuffer.HIGH_PRECISION = true.
Paulo > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On > Behalf Of Adam Goode > Sent: Thursday, December 15, 2005 5:13 AM > To: [email protected] > Subject: [iText-questions] Question about ByteBuffer.formatDouble > > Hi, > > I am using iText 1.3.6 to do some drawing in PDF. I was wondering why > ByteBuffer.formatDouble does some things, (and why it appears to > reimplement something like java.text.DecimalFormat). > > Most things seem ok, but why if (1.0 <= d <= 32767) then precision is > truncated to 2 decimal places? It looks like it is causing visual > artifacts in my drawing. For example, I am trying to scale a > part of my > drawing by 2.00676 and formatDouble outputs this as "2.01". > It is quite > noticeable since it causes two rectangles next to each other > to not line > up. > > If I manually change the number in the PDF output to the more precise > number, then everything lines up correctly. There doesn't seem to be a > reason for truncating the digits that much. > > On page 920 of the PDF standard version 1.6, Table C.1 says that real > numbers have approximately 5 significant decimal digits of precision, > which sounds about right given that previous versions of Acrobat used > 16.16 fixed point math. (New versions use IEEE float.) > > To fix this, I propose to change formatDouble to never output > less than > 6 places after the decimal point (except for trailing 0). Readers that > implement real numbers as 16.16 fixed point (like old Acrobat reader) > will be able to handle this precision. Newer versions of Acrobat (with > IEEE float) can also handle the precision, so it is not a waste to > output this. > > Along with this, why not use DecimalFormat? It looks like it would > eliminate most of the code in formatDouble. > > I can cook up a patch to fix this, with or without DecimalFormat. > > > Thanks, > > Adam > > ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_idv37&alloc_id865&op=click _______________________________________________ iText-questions mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/itext-questions
