rzo1 commented on code in PR #2075:
URL: https://github.com/apache/stormcrawler/pull/2075#discussion_r3879125142
##########
core/src/main/java/org/apache/stormcrawler/util/CookieConverter.java:
##########
@@ -129,6 +168,16 @@ public static List<Cookie> getCookies(String[]
cookiesStrings, URL targetURL) {
return list;
}
+ /** Checks whether a cookie domain is made of a single label, e.g. "com".
*/
+ private static boolean isSingleLabel(String domain) {
+ String d = domain;
+ if (d.startsWith(".")) {
+ d = d.substring(1);
+ }
+ int dot = d.indexOf('.');
+ return dot < 1 || dot == d.length() - 1;
Review Comment:
Follow-up: the hand-rolled normalisation is gone in favour of `IDN.toASCII`,
which is what crawler-commons calls internally
(`EffectiveTldFinder.normalizeName` and `BasicURLNormalizer` are both private,
so there is nothing public to reuse directly). It rejects the empty-label
family itself — `com..`, `com...`, `..example.com`, `some..url.com` all throw —
and makes the unicode and punycode forms interchangeable, so
`Domain=münchen.de` now matches `xn--mnchen-3ya.de` either way round. Also
closed the address case while here: an address is matched only by itself per
RFC 6265 5.1.3, where `Domain=2.3.4` set by `1.2.3.4` used to reach `9.2.3.4`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
