Yes please. I tried the oldest file from PDFBOX-3333, but it's rotated so there 
is no nice text extraction by default, but I saw the difference.

We don't need a PDF for the patch (but I'd like to have a real world PDF for 
me), the test can work with toUnicode() alone.

If you need to register to JIRA mention something so we know it's you, or use 
the same name.

Tilman

On 2026-11-17 13:13 UTC Michael Aleythe, Sternwald wrote:
> Hi all,
> 
> we extract text from newspaper PDFs with Tika 4.2.0 / PDFBox 3.0.8 and lose
> every word that contains an ff, fi, fl or ffi ligature when the font names
> these glyphs in the AGL underscore form (f_f, f_i, f_l, f_f_i) and has no
> /ToUnicode. PDFBox logs "No Unicode mapping for f_i (31) in font ..." and
> drops the glyph; Tika writes U+FFFD in its place. "Schaffhauser" becomes
> "Scha?hauser", "flexible" becomes "?exible", and the words no longer match
> in search.
> 
> The cause is GlyphList.toUnicode(): "fi" maps to U+FB01, "five.lin" maps via
> the dot-suffix rule, but "f_i" and "f_f_i" return null. PDFBox ships
> Adobe's glyph list and already applies the AGL specification's [1] suffix
> rule and uniXXXX/uXXXX forms; the underscore rule from the same section
> (f_f_i -> f f i) is the only part missing. trunk and 3.0 have the same
> code.
> 
> This has come up before as a warning (dev@ 2015 with HelveticaNeueLTStd,
> PDFBOX-3333 with MyriadPro-Bold, users@ 2019 with Gotham), and the answer
> then was that such PDFs are at fault. Strictly, that's right: ToUnicode is
> optional, and ISO 32000-1 9.10.2 only guarantees the glyph-name fallback
> for names from the Adobe standard Latin character set, which f_i is not.
> But PDFBox already goes beyond 9.10.2 with uniXXXX/uXXXX names and suffix
> stripping, and the underscore form is what section 6 of the AGL
> specification recommends for ligatures ("the 'o f f i' ligature should be
> named 'o_f_f_i'"). These are mainstream commercial fonts, and poppler
> (pdftotext) and pdf.js extract the same files correctly.
> 
> The change is small and only touches names that map to null today:
> 
>     else if (name.indexOf('_') >= 0)
>     {
>         // AGL specification: a ligature name maps component by component
>         unicode = toUnicodeComponents(name.split("_", -1));
>     }
> 
>     private String toUnicodeComponents(String[] components)
>     {
>         StringBuilder sb = new StringBuilder();
>         for (String component : components)
>         {
>             String part = component.isEmpty() ? null : toUnicode(component);
>             if (part == null)
>             {
>                 return null;
>             }
>             sb.append(part);
>         }
>         return sb.toString();
>     }
> 
> With it: f_i -> "fi", f_f_i -> "ffi", f_f_i.alt -> "ffi", T_h -> "Th",
> uni0066_uni0069 -> "fi"; f__i, _, f_ and f_bogus stay null; fi and A are
> unchanged. On a 12-page newspaper supplement it repaired 45 words and
> changed nothing else.
> 
> One choice: the AGL spec maps an unknown component to an empty string and
> keeps the rest. I return null for the whole name instead, so no output
> changes for names PDFBox can't resolve today. Happy to do it either way.
> 
> If this is welcome, I'll open a Jira issue with a minimal synthetic
> reproducer PDF (non-embedded Helvetica, /Differences with f_i, f_f_i, f_l,
> f_f, no ToUnicode) and a PR against trunk with a regression test, for
> backport to 3.0.
> 
> [1] https://github.com/adobe-type-tools/agl-specification
> 
> Thanks,
> Michael
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to