The GitHub Actions job "Java CI with Maven" on stormcrawler.git/fix-2091-reserved-metadata-keys has failed. Run started by GitHub user rzo1 (triggered by rzo1).
Head commit for run: 94c5c67313c998ac05790dba1794ad3785a1082a / Richard Zowalla <[email protected]> #2091 Protocols: reject metadata keys arriving in a response header The protocols copy every response header into the response metadata, and a header name is not restricted to anything, so a fetched server could write the keys the crawler uses to record its own fetch. WARCRecordFormat promotes a record from resource to response whenever _response.headers_ is non-blank and embeds the block verbatim, and WARCRequestRecordFormat writes _request.headers_ as what the crawler supposedly sent, so the archive could be made to contain a request and response of the server's choosing, with its own WARC-IP-Address and WARC-Date. A value under metrics.* reached the fetcher metric registry the same way, which is the remaining half of #2099. ProtocolResponse.isReservedMetadataKey now names the keys the crawler writes itself, and the okhttp and Playwright protocols skip a header which matches. In okhttp the keys HTTPHeadersInterceptor passes through the response headers stay readable, but only while that interceptor is installed, which is what http.store.headers does; the interceptor also drops the cipher suite header it does not set over http://, which was otherwise left for the server to choose. This also removes an unguarded base64 decode: a value under _request.headers_ which was not valid base64 threw IllegalArgumentException out of getProtocolOutput and failed the fetch. Report URL: https://github.com/apache/stormcrawler/actions/runs/35388246809 With regards, GitHub Actions via GitBox
