The GitHub Actions job "Java CI with Maven" on 
stormcrawler.git/issue-2085-spout-scheme-check has failed.
Run started by GitHub user abhinav-phi (triggered by rzo1).

Head commit for run:
44bb12e320c4e1e095af620578dd8ef685f35261 / abhinav-phi <[email protected]>
Spouts check the scheme of stored URLs before emitting (#2085)

The frontier is the crawl instruction set: whatever ends up in the
store is fetched, with no scheme check and no filtering, since URL
filtering only runs on the discovery path. A row whose URL uses a
scheme the operator never intended to crawl was emitted as long as a
protocol implementation was registered for it.

AbstractQueryingSpout.nextTuple now checks the scheme of each buffered
URL against the configured protocols list before emitting, skips rows
with other schemes at WARN with a skipped.scheme counter, and keeps
draining the buffer so one bad row does not block the ones behind it.

Report URL: https://github.com/apache/stormcrawler/actions/runs/33996885403

With regards,
GitHub Actions via GitBox

Reply via email to