rzo1 opened a new pull request, #2257: URL: https://github.com/apache/stormcrawler/pull/2257
Hardening, no known issue behind it. Core now creates its JAXP factories through [Apache Commons Secure XML](https://commons.apache.org/proper/commons-secure-xml/) (1.1.0, no runtime dependencies): - `jsoup.XPathFilter` and `parse.filter.XPathFilter` use `SecureXPathFactory.newDefaultInstance()`. This pins the JDK's XPath implementation instead of whatever is registered on the classpath, and turns on secure processing (no extension functions, JDK processing limits). The expressions still run against the in-memory DOM, so results are unchanged. - `RegexURLNormalizer` reads its rules file with `SecureDocumentBuilderFactory.newDefaultInstance()`, so external DTDs and entities are no longer fetched. Before, a rules file declaring an external DTD made the normalizer load no rules if the DTD could not be fetched. `RegexURLNormalizerTest` covers this; it fails on main. Left out on purpose: - `parse.filter.LDJsonParseFilter` and `RobotsTags`, removed in #2253. - The hardened `SAXParserFactory` in the Tika `ParserBolt`, which already rejects any DOCTYPE and so is stricter than the library default. That file is also changed by #2256, #2236 and #2052. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
