Common Crawl stores the payload uncompressed and unchunked to leverage processing of WARC files by users on various platforms on programming languages. To avoid potential errors by the WARC processors this requires to remove resp. change the HTTP headers "Content-Length", "Content-Encoding" and "Transfer-Encoding".
Now I want to preserve the original headers in a safe but transparent way. Is the preservation of original and rewritten HTTP headers in WARC files a valid use case for the X-Archive-Orig- header prefix, or is it thought only for wayback machines when serving captures over HTTP? Thanks, Sebastian -- You received this message because you are subscribed to the Google Groups "openwayback-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/openwayback-dev. To view this discussion on the web visit https://groups.google.com/d/msgid/openwayback-dev/371f84db-0486-4f7d-860a-a37d7d3c06df%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
