jnioche commented on PR #2005: URL: https://github.com/apache/stormcrawler/pull/2005#issuecomment-5034944827
> Thanks for the context @jnioche . I understand that ParseFilters are used across all parsers, including SiteMapParserBolt. > > I followed the approach you suggested in the issue — using a minimalistic HTML document instead. The rewritten SubDocumentsParseFilter now works with the W3C DOM (DocumentFragment) provided by JSoupParserBolt, rather than re-parsing the raw bytes as XML. Sorry @AsifQureshi04, I opened the issue 10 years ago (!?!) and needed a bit of time to refresh my memory. Your PR is fine as it is. > Would you prefer I also add a separate test that demonstrates SubDocumentsParseFilter working with SiteMapParserBolt? Happy to adjust based on your feedback. That's what I was wondering about but actually it is probably not needed and would mean that the document would be reparsed as SitemapParserBolt does not generate a DOM. Thanks for your contribution to StormCrawler -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
