So are all the ‘garbage’ characters I’m seeing simply due to unmapped characters? Is there any solution to make it look prettier? Or is ‘flattening’ (if that’s the correct word), the best solution?
Peter Kronenberg | Senior AI Analytic ENGINEER C: 703.887.5623 [Torch AI]<http://www.torch.ai/> 4303 W. 119th St., Leawood, KS 66209 WWW.TORCH.AI<http://www.torch.ai/> From: Tilman Hausherr <[email protected]> Sent: Tuesday, December 21, 2021 10:06 AM To: [email protected] Subject: Re: Garbage in PDF files Am 21.12.2021 um 15:51 schrieb Peter Kronenberg: Running the attached PDF file through TIKA, I get a lot of garbage in the output (see txt file). Far more than can be explained by the unmapped characters. Where is this coming from? After the character is found to be unmapped, PDFBox tries a backup strategy, which is obviously not successful here. // when there is no Unicode mapping available, Acrobat simply coerces the character code // into Unicode, so we do the same. Subclasses of PDFStreamEngine don't necessarily want // this, which is why we leave it until this point in PDFTextStreamEngine. if (unicode == null) { if (font instanceof PDSimpleFont) { char c = (char) code; unicode = new String(new char[] { c }); } else { // Acrobat doesn't seem to coerce composite font's character codes, instead it // skips them. See the "allah2.pdf" TestTextStripper file. return; } } Adobe Reader has also trash. I can't comment whether it is "far more than expected", this would require to count and make comparisons. If I take the PDF and flatten it by ‘printing’ to a PDF file, the garbage goes away Printing is probably converting it all to raster graphics or vector graphics. Tilman
