AsifQureshi04 commented on issue #158: URL: https://github.com/apache/stormcrawler/issues/158#issuecomment-5036463311
Hi @jnioche , I'd like to work on this issue. After exploring the codebase, I found that `needsDOM()` is used in two places: 1. `JSoupParserBolt` — to skip building the W3C DocumentFragment from the JSoup Document 2. Tika's `ParserBolt` — to skip adding a DOMBuilder to the handler chain Three production filters override it to return `true` (XPathFilter, LDJsonParseFilter, DebugParseFilter), and ParseFilters.filter() uses it to skip filters that need DOM when none is available (e.g., from SiteMapParserBolt). Could you confirm the preferred approach? 1. **Remove `needsDOM()` entirely** and always build the DOM in both parsers 2. **Remove only the method from ParseFilter's public API** but keep the lazy DOM construction internally (always build it when the parser has the content available) Happy to implement whichever direction you prefer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
