I understand your concerns, but I think I know pretty well "what I am doing" :)
For example, if you upload an gzipped file to an S3 bucket and then retrieve it, it also won't automatically come with the right Content-Encoding header. On AWS, however, one can set it up in a way that the correct Content-Encoding header is returned by manually adding the headers to the object. However, in my situation I have no way of setting up things so that the proper header is returned (but I do know that the file is gzipped properly, since I uploaded it). But yeah, maybe the better solution is to unzip it "manually" (without libcurl). Best, Wolf Am Do., 7. Juli 2022 um 11:40 Uhr schrieb Timothe Litt via curl-library < curl-library@lists.haxx.se>: > It sounds like you're on the wrong track. > > I'm not familiar with that service. I would expect it to return what you > provide. > > If you upload gzip'd data, the server should return gzip'd data - it's a > blob. This makes sense if you want to minimize stored size on the server. > > If you want to upload an uncompressed binary blob and get back the > uncompressed data, but use gzip on the wire - consider transfer-encoding. > This will store the uncompressed data on the server - which may be what > you're trying to avoid. > > Did you use the correct content-encoding (or transfer-encoding) headers > when you uploaded the blobs? > > Are you specifying the correct accept-encoding header on the download? > > As for changing incoming headers - I don't think that's doable, but in any > case, it's a bad idea. "There be dragons there." > > curl uses zlib for encoding/decoding, so aside from convenience, you would > get nothing from forcing a path thru the internal functions. > > > Timothe Litt > ACM Distinguished Engineer > -------------------------- > This communication may not represent the ACM or my employer's views, > if any, on the matters discussed. > > On 07-Jul-22 05:19, Wolf Vollprecht wrote: > > Hi Timothe, > > Thanks for your message. Indeed, I want to decode the data. The server is > not broken (it’s the Github OCI registry server), I just cannot control > what headers it sends for the binary blobs that I uploaded. > > Ideally I would have wanted to re-use the internal libcurl functions for > decoding that would be used if the server would reply with the correct > Content-Encoding vs implementing it in my WRITEFUNCTION. > > Best, > > Wolf > > > On 7. Jul 2022, at 11:15, Timothe Litt via curl-library < > curl-library@lists.haxx.se> wrote: > > > On 07-Jul-22 03:56, Wolf Vollprecht via curl-library wrote: > > Hi all, > > I have some data that I am downloading where I know that it is gzip-encoded. > However, the server does not respond with the correct header (and I don’t > have control over it). I was wondering if there is a way to either “manually” > append the proper Content-Encoding header to the response or force libcurl to > encode the response with gzip? > > Best, > > Wolf > > I assume you mean "decode", not "encode". After the mandatory lecture > about not tolerating broken servers... > > For the command line: > > curl https://foo.example.com/bar | gunzip > data > > If you are using libcurl, depending on your requirements see zlib (libz) > with CURLOPT_WRITEFUNCTION or popen() gunzip (or gzip -d). > > > Timothe Litt > ACM Distinguished Engineer > -------------------------- > This communication may not represent the ACM or my employer's views, > if any, on the matters discussed. > > > -- > Unsubscribe: https://lists.haxx.se/listinfo/curl-library > Etiquette: https://curl.se/mail/etiquette.html > > > -- > Unsubscribe: https://lists.haxx.se/listinfo/curl-library > Etiquette: https://curl.se/mail/etiquette.html >
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html