AW: Does (or will?) HttpClient handle compressed content ?

[Ingo Meyer]

Hi all,

I agree with the point that encoding should be a part of HttpClient. 
But for all who don't want to wait until it's implemented here my solution:

First, add this to your HostConfiguration:

                final Collection<Header> c = new ArrayList<Header> ();
                c.add (new Header ("Accept",
"text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q
=0.8,image/png,*/*;q=0.5"));
                c.add (new Header ("Accept-Language",
"de-de,de;q=0.8,en-us;q=0.5,en;q=0.3"));
        -->     c.add (new Header ("Accept-Encoding", "gzip, deflate"));
                c.add (new Header ("Accept-Charset",
"ISO-8859-1,utf-8;q=0.7,*;q=0.7"));
                hostConfiguration.getParams ().setParameter
("http.default-headers", c);


Secondly, after the transfer is proceeded call the following method with the
response bytes:

        protected byte[] decodeResponse (final byte[] responseBytes) throws
Exception
        {
                LOG.debug ("request enter decodeResponse()");

                // Getting Content-Encoding
                final String sContentEncoding = getResponseHeader
("Content-Encoding");

                // If available...
                if (sContentEncoding != null)
                {
                        InputStream encoded = null;

                        // GZIP
                        if (sContentEncoding.equals ("gzip"))
                        {
                                LOG.debug ("method received gzip-encoded
content");
                                
                                encoded = new GZIPInputStream (new
ByteArrayInputStream (responseBytes));
                        }
                        // DEFLATE
                        else if (sContentEncoding.equals ("deflate"))
                        {
                                LOG.debug ("method received deflate-encoded
content");
                                
                                encoded = new InflaterInputStream (new
ByteArrayInputStream (responseBytes));
                        }
                        
                        final ByteArrayOutputStream decoded = new
ByteArrayOutputStream ();
                        final byte buffer[] = new byte[1024];

                        for (int length; (length = encoded.read (buffer, 0,
1024)) != -1;)
                        {
                                decoded.write (buffer, 0, length);
                        }

                        // closing
                        encoded.close ();
                        decoded.close ();

                        return decoded.toByteArray ();
                }
                
                LOG.debug ("method received uncoded content");
                        
                return responseBytes;
        }


If anyone want's to use it one thing still not work. The deflate part!
I have no idea about it, all works fine with gzip. So to get it running
better
change to just  gzip:

        -->     c.add (new Header ("Accept-Encoding", "gzip"));

So any help with the deflate part will be recommened.

Thanks 

Ingo Meyer



> -----Ursprüngliche Nachricht-----
> Von: Roland Weber [mailto:[EMAIL PROTECTED] 
> Gesendet: Samstag, 10. Dezember 2005 21:29
> An: HttpClient User Discussion
> Betreff: Re: Does (or will?) HttpClient handle compressed content ?
> 
> RFC 2616 distinguishes between content codings (section 3.5) 
> and transport codings (section 3.6). Content coding is out of 
> scope, as Oleg mentioned. HttpClient will not automatically 
> decompress for example a gzipped tar into a plain tar archive.
> 
> Transport coding is in scope for HttpClient/HttpComponents, 
> as we already support "chunked". We can consider adding 
> support for other transport codings, such as "gzip". The 
> pluggable filter architecture of HttpComponents will at least 
> allow you to add your own support for transport codings. I 
> guess that seamless interoperability of the HttpComponents 
> "chunked" implementation with additional custom transport 
> encodings will be made possible on request.
> I don't think we'll find time to add new transport codings 
> out of the box in HttpClient nor HttpComponents. Which 
> implies that the interaction of chunked with other transport 
> codings will not be tested until somebody tries to do it. It 
> is a tricky thing, since the response filter chain will have 
> to depend on the value of the Transfer-Encoding response 
> header. Given the complexity of the task, I see this item 
> dangling near the end of the to-do list forever.
> As always, contributions will be welcome.
> 
> If somebody should contribute code that performs 
> decompression for transport codings, we sure won't stop you 
> from re-using that code for decompression of content codings. 
> But that will remain the responsibility of the application 
> using HttpComponents.
> 
> cheers,
>   Roland
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: 
> [EMAIL PROTECTED]
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]