The GitHub Actions job "Java CI with Maven" on stormcrawler.git/issue-2083-sitemapparser-strict has failed. Run started by GitHub user abhinav-phi (triggered by dpol1).
Head commit for run: f880c11ba6e9256d784bed683a9a3cc0f030c668 / abhinav-phi <[email protected]> Sitemap strict checking defaults to off; sniffing confirms declared sitemaps (#2083) Review feedback on #2125: - the strict flag of crawler-commons is strict URL checking (a sitemap only yields URLs below its own host and path), not the namespace check the javadoc and yaml comment described. Fixed the wording and flipped the default to false: a sitemap living at example.com while listing URLs under www.example.com violates the sitemap spec but is common, and switching strict checking on by default silently shrinks such crawls. Recommended for open crawls, documented in the yaml - the sniff override no longer happens only for unmarked documents: a document already marked isSitemap=true has the namespace sniffed to confirm its type and gets ct forced to application/xml, so a sitemap served as text/html parses again instead of failing with UnknownFormatException - this was a regression against main - a test asserts the default keeps parsing cross-host sitemaps, and one asserts the wrong-content-type sitemap still parses Report URL: https://github.com/apache/stormcrawler/actions/runs/34151185538 With regards, GitHub Actions via GitBox
