Hi Jonas,
The fix for the xn--9tfky.id public suffix matching issue you reported
is merged into master and backported to 5.6.x. Feel free to check it
against your own patch/tests.
Thanks for the report.
On 11/19/26 7:26 AM, Jonas Mandalunes wrote:
Hello,
PublicSuffixMatcher in HttpClient 5.6.5, 5.7-beta1, and 4.5.14 does not
match the Public Suffix List rule for the Balinese name xn--9tfky.id when
the host is in ASCII form. As a result, PublicSuffixDomainFilter lets a
cookie with Domain=xn--9tfky.id reach a host in another site. A patch of
five changed lines is below. I would have opened a JIRA issue, but public
sign-up for the ASF JIRA is disabled, so I report it here.
The Public Suffix List contains the rule U+1B29 U+1B2E U+1B36 + ".id"
(A-label form xn--9tfky.id). The list bundled with 5.6.5 (dated 2026-10-06)
holds it. With the A-label form of the name, PublicSuffixMatcher does not
match the rule. Balinese was added in Unicode 5.0, and java.net.IDN
implements IDNA2003 on Unicode 3.2, so IDN.toUnicode returns the label
unchanged and the lookup in the Unicode rule table fails.
Reproduction without any list file (JDK 21.0.13, same output with 5.6.5,
5.7-beta1, and 4.5.14):
PublicSuffixMatcher m = new PublicSuffixMatcher(Arrays.asList("id",
"\u1B29\u1B2E\u1B36.id"), Collections.<String>emptyList());
System.out.println(m.getDomainRoot("q.xn--9tfky.id")); // actual:
xn--9tfky.id, expected: q.xn--9tfky.id
System.out.println(m.matches("xn--9tfky.id")); // actual:
false, expected: true
System.out.println(m.getDomainRoot("q.\u1B29\u1B2E\u1B36.id")); // Unicode
form: q.<Balinese>.id, correct
With the list bundled in the 5.6.5 jar
(PublicSuffixMatcherLoader.getDefault()), the same calls give the same
result. In addition, verify("xn--9tfky.id") returns true, and
PublicSuffixDomainFilter lets a cookie with Domain=xn--9tfky.id match the
origin b.r.xn--9tfky.id, which is a host in another site. That is the check
the public suffix list is used for.
Related change: the commit of 24 August 2026 (pull request 876, "decode
punycode public suffixes in PublicSuffixMatcher.verify") decodes A-labels
in verify. It fixes names such as xn--h-2fa.no, but it uses the same IDN
conversion (DnsUtils.normalizeUnicode), so this label still fails.
Suggested fix
Pass IDN.ALLOW_UNASSIGNED to the conversions. IDN.toUnicode alone is not
enough, because getDomainRoot then calls IDN.toASCII, which throws an
IllegalArgumentException for this label unless the flag is set (checked on
JDK 21.0.13). Version 5 needs three changed lines and version 4.5.x two:
---
a/httpclient5/src/main/java/org/apache/hc/client5/http/psl/PublicSuffixMatcher.java
+++
b/httpclient5/src/main/java/org/apache/hc/client5/http/psl/PublicSuffixMatcher.java
@@ -141,12 +141,12 @@
String normalized = DnsUtils.normalize(domain);
final boolean punyCoded = normalized.contains("xn-");
if (punyCoded) {
- normalized = IDN.toUnicode(normalized);
+ normalized = IDN.toUnicode(normalized, IDN.ALLOW_UNASSIGNED);
}
final DomainRootInfo match = resolveDomainRoot(normalized,
expectedType);
String domainRoot = match != null ? match.root : null;
if (domainRoot != null && punyCoded) {
- domainRoot = IDN.toASCII(domainRoot);
+ domainRoot = IDN.toASCII(domainRoot, IDN.ALLOW_UNASSIGNED);
}
return domainRoot;
}
---
a/httpclient5/src/main/java/org/apache/hc/client5/http/utils/DnsUtils.java
+++
b/httpclient5/src/main/java/org/apache/hc/client5/http/utils/DnsUtils.java
@@ -84,7 +84,7 @@
}
String decoded;
try {
- decoded = IDN.toUnicode(s);
+ decoded = IDN.toUnicode(s, IDN.ALLOW_UNASSIGNED);
} catch (final IllegalArgumentException ignore) {
decoded = s;
}
---
a/httpclient/src/main/java/org/apache/http/conn/util/PublicSuffixMatcher.java
+++
b/httpclient/src/main/java/org/apache/http/conn/util/PublicSuffixMatcher.java
@@ -141,7 +141,7 @@
String result = null;
while (segment != null) {
// An exception rule takes priority over any other matching
rule.
- final String key = IDN.toUnicode(segment);
+ final String key = IDN.toUnicode(segment,
IDN.ALLOW_UNASSIGNED);
final DomainType exceptionRule = findEntry(exceptions, key);
if (match(exceptionRule, expectedType)) {
return segment;
@@ -158,7 +158,7 @@
final String nextSegment = nextdot != -1 ?
segment.substring(nextdot + 1) : null;
if (nextSegment != null) {
- final DomainType wildcardDomainRule = findEntry(rules,
"*." + IDN.toUnicode(nextSegment));
+ final DomainType wildcardDomainRule = findEntry(rules,
"*." + IDN.toUnicode(nextSegment, IDN.ALLOW_UNASSIGNED));
if (match(wildcardDomainRule, expectedType)) {
if (wildcardDomainRule == DomainType.PRIVATE) {
return segment;
*Verification*
I generated 35,115 queries from the list of 2 September 2026 (every rule,
its parent, and every right-anchored suffix of hosts below it), ran them
through the stock classes and through the patched classes (patched classes
in front of the released jar), and compared the answers with the PSL
algorithm, with the published file and ASCII hosts. Version 5.6.5: 4
differing answers before the patch, 0 after it. Version 4.5.14: 6 before, 2
after. The 2 that remain in 4.5.14 do not concern internationalized names.
No answer changed in the wrong direction.
I file this as a *bug and not as a vulnerability*. The cause is the
documented IDNA2003 behavior of java.net.IDN, and the HttpComponents
security model asks reporters not to file behavior that is legitimate for
the JDK as a vulnerability. Please tell me if you want it handled
differently, or if you prefer that I open a JIRA issue once I have an
account.
Kind regards,
Jonas Mandalunes
*Researcher*
AMA Computer University, Graduate School, Quezon City, Philippines
[email protected]
ORCID 0009-0003-4770-8540
--
Arturo
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]