[ 
https://issues.apache.org/jira/browse/PDFBOX-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18147720#comment-18147720
 ] 

Martin Mirchev commented on PDFBOX-5126:
----------------------------------------

The surrogate-pair part of this is still present on trunk: 
{{PDFTextStripper.handleDirection()}} reverses RTL runs one {{char}} at a time, 
so a supplementary character in an RTL run comes out as low surrogate followed 
by high surrogate. A simple case is an emoji between Hebrew words: the attached 
{{rtl-emoji.pdf}} (Helvetica with a ToUnicode CMap mapping to ืฉืœื•ื ๐Ÿ˜€ ืขื•ืœื) 
extracts as {{... 05E2 0020 DE00 D83D 0020 ...}}.

Stepping back by code point keeps the pairs in order; with this change the file 
extracts as {{... 05E2 0020 D83D DE00 0020 ...}}, and LTR text and 
supplementary characters in LTR runs are unchanged. It does not address the 
combining-mark part of this issue.

{code}
--- a/pdfbox/src/main/java/org/apache/pdfbox/text/PDFTextStripper.java
+++ b/pdfbox/src/main/java/org/apache/pdfbox/text/PDFTextStripper.java
@@ -1949,23 +1949,19 @@
 
             if ((level & 1) != 0)
             {
-                while (--end >= start)
+                while (end > start)
                 {
+                    // step back by code point so that surrogate pairs stay in 
order
+                    int codePoint = word.codePointBefore(end);
+                    end -= Character.charCount(codePoint);
                     char character = word.charAt(end);
-                    if (Character.isMirrored(word.codePointAt(end)))
+                    if (Character.isMirrored(codePoint) && 
MIRRORING_CHAR_MAP.containsKey(character))
                     {
-                        if (MIRRORING_CHAR_MAP.containsKey(character))
-                        {
-                            result.append(MIRRORING_CHAR_MAP.get(character));
-                        }
-                        else
-                        {
-                            result.append(character);
-                        }
+                        result.append(MIRRORING_CHAR_MAP.get(character));
                     }
                     else
                     {
-                        result.append(character);
+                        result.appendCodePoint(codePoint);
                     }
                 }
             }
{code}

This was found by an LLM-assisted tool and verified manually.


> Complex Unicode glyphs (surrogate pairs, combining diacritics, zero-width 
> join, etc.) in a RTL context get reversed incorrectly on text extraction
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PDFBOX-5126
>                 URL: https://issues.apache.org/jira/browse/PDFBOX-5126
>             Project: PDFBox
>          Issue Type: Bug
>          Components: Text extraction
>    Affects Versions: 2.0.22
>            Reporter: Gรกbor Stefanik
>            Priority: Major
>         Attachments: rovasvegyes.pdf, rtl-emoji.pdf
>
>
> The attached PDF contains old Hungarian runic script, which is both 
> right-to-left and outside Unicode's Basic Multilingual Plane (and thus 
> encoded as surrogate pairs in Java's internal UTF-16-like representation). 
> When this text is extracted, the surrogate pairs are reversed due to an 
> overly naive use of "char"-level reversal, leading to malformed Unicode 
> output.
> Likewise, when combining diacritics/modifiers occur in a right-to-left 
> context, their position relative to the "parent" character is reversed, and 
> so they appear on the wrong glyph, as demonstrated by the Hebrew sample in 
> the same PDF. I imagine the same thing would also happen to emoji using the 
> "zero-width joiner" in an RTL context.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to