rzo1 opened a new pull request, #2165: URL: https://github.com/apache/stormcrawler/pull/2165
Fixes #2095. Depends on #2158 (stacked; review only the last commit until #2158 is merged). The existing route handler now checks each browser request (page, redirects, subresources) against `http.filter.ipaddress.*` and aborts rejected ones. Not applied through `http.proxy`, same as OkHttp. The address is resolved separately from the browser and WebSockets are not checked; both limits are documented. URL filters are not applied here: main page redirects are already aborted and go through the regular pipeline, and crawl-scope filters on subresources would break rendering. **Release note:** with the default exclude list, Playwright topologies crawling loopback or private addresses need to override `http.filter.ipaddress.exclude`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
