rzo1 commented on code in PR #2075:
URL: https://github.com/apache/stormcrawler/pull/2075#discussion_r3879003625
##########
core/src/main/java/org/apache/stormcrawler/util/CookieConverter.java:
##########
@@ -129,6 +168,16 @@ public static List<Cookie> getCookies(String[]
cookiesStrings, URL targetURL) {
return list;
}
+ /** Checks whether a cookie domain is made of a single label, e.g. "com".
*/
+ private static boolean isSingleLabel(String domain) {
+ String d = domain;
+ if (d.startsWith(".")) {
+ d = d.substring(1);
+ }
+ int dot = d.indexOf('.');
+ return dot < 1 || dot == d.length() - 1;
Review Comment:
Replaced the label check with normalisation plus the crawler-commons PSL.
Empty labels are kept when splitting, so `com..`, `com...` and `.com..` are
rejected instead of matching every host under `com`, and a domain which is
itself a suffix (`com`, `co.uk`, `pvt.k12.ma.us`, `github.io` from the private
section, the `*.ck` wildcard) can no longer cover subdomains — it is ignored
and the cookie binds to the host, per RFC 6265 5.3. Domains the list does not
know, `.local` and `.internal` included, fall back to the plain suffix match
and are covered by tests, as is a single-label intranet domain.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
