The GitHub Actions job "Java CI with Maven" on stormcrawler.git/issue-2088-redirect-filtering has failed. Run started by GitHub user abhinav-phi (triggered by rzo1).
Head commit for run: de47d5321a3eeecb8d40f648ecca732a0b656383 / abhinav-phi <[email protected]> Run redirect hops through the URL filters (#2088) With http.allow.redirects enabled, okhttp followed the whole redirect chain inside the client and the crawler only ever saw the final response, returned under the original URL: no intermediate target went through the URL filters, so the scheme exclusions, host confinement and depth rules configured for the crawl did not apply to the hops a fetched page steered the fetcher to. Redirects are now followed manually in getProtocolOutput: each target is resolved, run through the URL filter chain and only fetched when accepted; a rejected target ends the chain and the redirect response is returned as is, which the caller handles like it does when immediate follows are off. The chain is capped at 5 hops, and the final URL is recorded in the response metadata under _redirTo so callers can tell the content is not from the URL they asked for. Report URL: https://github.com/apache/stormcrawler/actions/runs/34019929142 With regards, GitHub Actions via GitBox
