Tilman Hausherr created PDFBOX-6210:
---------------------------------------
Summary: Incorrect CJK Character Extraction for Shared Glyphs
Key: PDFBOX-6210
URL: https://issues.apache.org/jira/browse/PDFBOX-6210
Project: PDFBox
Issue Type: Bug
Affects Versions: 3.0.7 PDFBox, 2.0.36
Reporter: Tilman Hausherr
Fix For: 2.0.37, 3.0.8 PDFBox, 4.0.0
As described by Chanhyuk Lee in attached PR
===
h2. Problem
When multiple Unicode code points map to the same glyph, text extracted from a
generated PDF may not match the character originally entered by the author.
This commonly occurs in CJK fonts, where a CJK Unified Ideograph and its
radical counterpart share a glyph. For example, in Noto Sans CJK, both 食
(U+98DF) and ⻝ (U+2EDD, CJK RADICAL EAT ONE) map to the same glyph. As a
result, rendering 食 in a PDF and extracting it with PDFTextStripper may
incorrectly produce ⻝.
h2. Root Cause
PDCIDFontType2Embedder.buildToUnicodeCMap() constructs the /ToUnicode map by
reverse-looking up a glyph's code point using:
cmapLookup.getCharCodes(gid).get(0)
However, getCharCodes() returns code points sorted in ascending order.
Consequently, the first entry is always the lowest code point associated with
the glyph. In the example above, U+2EDD is selected instead of U+98DF because
it has the smaller value.
As a result, the generated /ToUnicode mapping may point to a compatibility or
radical character rather than the character that was actually used in the
document. The existing comment ("use the first entry even for ambiguous
mappings") already acknowledges this limitation.
h5. This issue is not unique to PDFBox and has also been observed in
[wkhtmltopdf|https://github.com/wkhtmltopdf/wkhtmltopdf/issues/4414]/[Qt|https://github.com/qt/qtbase/blob/389988c42f901f2d8f75a023039d641cf5fba9de/src/gui/text/qfontsubset.cpp#L200-L209],
[Mozilla Firefox|https://bugzilla.mozilla.org/show_bug.cgi?id=1881196], and
[Typst|https://github.com/typst/typst/issues/4582].
h2. Fix
The code points actually used in the document are already tracked by
TrueTypeEmbedder.addToSubset(int).
This change builds a glyph → code point mapping from those recorded inputs and
uses it when generating the /ToUnicode CMap. For glyphs associated with
multiple code points, the first code point encountered in the document is
preferred.
The existing reverse cmap lookup is retained only as a fallback for glyphs that
have no recorded input code point (for example, glyphs drawn directly by GID).
To ensure deterministic behavior, subsetCodePoints is changed from a HashSet to
a LinkedHashSet, preserving insertion order and making the "first occurrence
wins" rule stable.
This follows the same approach adopted by Typst to resolve the identical issue
([typst/typst#4582|https://github.com/typst/typst/issues/4582], fixed in
[typst/typst#4585|https://github.com/typst/typst/pull/4585]): prefer the code
point that was actually used rather than reverse-mapping from the font cmap.
h2. Tests
Two tests were added to TestFontEmbedding using Noto Sans CJK KR:
h4. testToUnicodePrefersUsedCodePoint
Scans the font for any glyph shared by multiple printable code points
(preferably a CJK ideograph/radical pair) and verifies that each code point
round-trips through PDF generation and extraction as itself.
h4. testToUnicodeCjkAndRadicalLookAlike
Uses the explicit pair 食 (U+98DF) and ⻝ (U+2EDD) and verifies that:
both characters share the same glyph,
the radical is the first entry returned by the cmap reverse lookup,
an ideograph entered as 食 is extracted as 食, and
an intentionally entered radical ⻝ is preserved as ⻝.
Before this change, the test fails because 食 is extracted as ⻝.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
