jnioche opened a new pull request, #2011: URL: https://github.com/apache/stormcrawler/pull/2011
## Problem `JSoupParserBolt` resolved relative outlinks against the document URL, ignoring any `<base href>` element in the page. For a page served from a sub-path whose `<base>` points at the domain root, this doubled the path segment. Example — page `https://host/fr/rayons/actualites-reportages/216943` with `<base href="https://host/";>` and a link `href="fr/rayons/stockage-archivage-de-donnees/218725"`: - Expected: `https://host/fr/rayons/stockage-archivage-de-donnees/218725` - Produced: `https://host/fr/rayons/actualites-reportages/fr/rayons/stockage-archivage-de-donnees/218725` ## Fix Resolve relative links against `jsoupDoc.baseUri()` instead of the raw document URL. jsoup updates `baseUri()` to the `<base href>` value when the element is present, matching browser behaviour, and falls back to the document URL otherwise. This keeps the existing fast resolution path (no per-link `abs:href` object building). ## Test Added `testBaseHrefOutlinkResolution` reproducing the doubled-path case. It fails without the fix (producing the doubled URL) and passes with it. Full `JSoupParserBoltTest` class: 12/12 passing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
