[
https://issues.apache.org/jira/browse/TEXT-192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248247#comment-17248247
]
Bruno P. Kinoshita commented on TEXT-192:
-----------------------------------------
Hi [~cpars509]
Thanks for the well detailed issue report.
>Looking at the source for how StringEscapeUtils.UNESCAPE_HTML4 works, I think
>(big question marks here) that an escape set needs to be added for
>Windows-1252?
I believe you are correct. That field is created for ISO-8859-1. You could
probably create a similar object, but creating a new entry in EntityArrays for
Windows-1252. Maybe we could have that added via a PR, to support windows
encoding by default.
Cheers
Bruno
> HTML unescape does not parse Windows-1252 correctly
> ---------------------------------------------------
>
> Key: TEXT-192
> URL: https://issues.apache.org/jira/browse/TEXT-192
> Project: Commons Text
> Issue Type: Bug
> Affects Versions: 1.8
> Environment: Java, macOS; should not be platform specific
> Reporter: Cyril Parsons
> Priority: Minor
>
> Looking at [https://en.wikipedia.org/wiki/Windows-1252#Character_set] there
> are differences in parsing Windows-1252 and ISO 8859-1. Code points between
> 128 and 159 (on Windows-1252) are improperly decoded.
> In a MMVE:
> {code:java}
> import org.apache.commons.text.StringEscapeUtils;
> ...
> String w1252 = "~—¡";
> String output = StringEscapeUtils.unescapeHtml4(w1252);
> System.out.println(output);
> System.out.println(output.chars().mapToLong(Long::valueOf)
> .boxed().collect(Collectors.toList()));
> {code}
> The output is:
> {code:java}
> ~ ¡
> [126, 151, 161]
> {code}
> (Space substituted for the Unicode character "End Of Guarded Area".) Expected
> output would be that em-dash would appear. Code points right outside of the
> Windows-1252/ISO 8859-1 inconsistency zone are all okay. Looking at the
> source for how StringEscapeUtils.UNESCAPE_HTML4 works, I think (big question
> marks here) that an escape set needs to be added for Windows-1252?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
