[
https://issues.apache.org/jira/browse/HTTPCLIENT-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Lancaster updated HTTPCLIENT-1113:
----------------------------------------
I found a workaround, but it would be nice if this problem could be solved more
elegantly.
Since the file was being downloaded in its entirety, I added some code to
ignore the EOFException.
if (responseCode == HttpStatus.SC_OK && entity != null) {
outStream = new
BufferedOutputStream(new FileOutputStream(outFilename));
long length = entity.getContentLength();
try {
entity.writeTo(outStream);
} catch (EOFException e) {
// Sometimes we get
EOFExceptions if the content length
// wasn't specified by the
server. Ignore these but report
// those where the content
length was specified.
if (length >= 0) {
throw e;
}
}
}
> Error downloading text file with gzip content encoding
> ------------------------------------------------------
>
> Key: HTTPCLIENT-1113
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1113
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 4.1.1
> Environment: Ubuntu, Java 1.5
> Reporter: Peter Lancaster
> Priority: Critical
> Labels: gzip
>
> Hello I am getting an exception when I try to download certain files.
> I don't have control over the host server, only the client. Here's my client
> code:
> HttpParams params = new BasicHttpParams();
> params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
> 300000L);
> params.setParameter(ClientPNames.HANDLE_REDIRECTS, true);
> // This client indicates to servers that it will support 'gzip'
> // and 'deflate' compressed responses.
> ContentEncodingHttpClient.setDefaultHttpParams(params);
> ContentEncodingHttpClient client = new
> ContentEncodingHttpClient();
> if (user != null && password != null) {
> String hostname = url.getHost();
> HttpHost hostHttp = new HttpHost(hostname, 80, "http");
> HttpHost hostHttps = new HttpHost(hostname, 443,
> "https");
> client.getCredentialsProvider().setCredentials(
> new AuthScope(hostname, 80),
> new UsernamePasswordCredentials(user,
> password));
>
> client.getCredentialsProvider().setCredentials(
> new AuthScope(hostname, 443),
> new UsernamePasswordCredentials(user,
> password));
>
> // Create AuthCache instance
> AuthCache authCache = new BasicAuthCache();
> // Generate BASIC scheme object and add it to the local
> auth cache
> BasicScheme basicAuth = new BasicScheme();
> authCache.put(hostHttp, basicAuth);
> authCache.put(hostHttps, basicAuth);
>
> // Add AuthCache to the execution context
> BasicHttpContext localcontext = new BasicHttpContext();
> localcontext.setAttribute(ClientContext.AUTH_CACHE,
> authCache);
> }
> HttpGet httpget = new HttpGet(url.toString());
> httpget.setHeader("If-Modified-Since", lastModified);
> HttpResponse response = client.execute(httpget);
> responseCode = response.getStatusLine().getStatusCode();
> HttpEntity entity = response.getEntity();
> if (responseCode == HttpStatus.SC_NOT_MODIFIED) {
>
> } else if (responseCode == HttpStatus.SC_OK && entity != null) {
> outStream = new BufferedOutputStream(new
> FileOutputStream(outFilename));
> entity.writeTo(outStream);
> }
> Here's the log output:
> DEBUG [2011-08-02 01:23:01,031]
> [org.apache.http.impl.conn.SingleClientConnManager:212] Get connection for
> route HttpRoute[{}->http://<host>]
> DEBUG [2011-08-02 01:23:01,036]
> [org.apache.http.impl.conn.DefaultClientConnectionOperator:145] Connecting to
> <host>/<IP>:80
> DEBUG [2011-08-02 01:23:01,057]
> [org.apache.http.client.protocol.RequestAddCookies:132] CookieSpec selected:
> best-match
> DEBUG [2011-08-02 01:23:01,057]
> [org.apache.http.client.protocol.RequestAuthCache:75] Auth cache not set in
> the context
> DEBUG [2011-08-02 01:23:01,058]
> [org.apache.http.impl.client.DefaultRequestDirector:631] Attempt 1 to
> execute request
> DEBUG [2011-08-02 01:23:01,058]
> [org.apache.http.impl.conn.DefaultClientConnection:264] Sending request: GET
> <file> HTTP/1.1
> DEBUG [2011-08-02 01:23:01,058] [org.apache.http.impl.conn.Wire:63] >>
> "GET <file> HTTP/1.1[\r][\n]"
> DEBUG [2011-08-02 01:23:01,058] [org.apache.http.impl.conn.Wire:63] >>
> "If-Modified-Since: Mon, 01 Aug 2011 18:26:09 CEST[\r][\n]"
> DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63] >>
> "Host: <host>[\r][\n]"
> DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63] >>
> "Connection: Keep-Alive[\r][\n]"
> DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63] >>
> "User-Agent: Apache-HttpClient/4.1.1 (java 1.5)[\r][\n]"
> DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63] >>
> "Accept-Encoding: gzip,deflate[\r][\n]"
> DEBUG [2011-08-02 01:23:01,060] [org.apache.http.impl.conn.Wire:63] >>
> "[\r][\n]"
> DEBUG [2011-08-02 01:23:01,060]
> [org.apache.http.impl.conn.DefaultClientConnection:268] >> GET <file> HTTP/1.1
> DEBUG [2011-08-02 01:23:01,060]
> [org.apache.http.impl.conn.DefaultClientConnection:271] >> If-Modified-Since:
> Mon, 01 Aug 2011 18:26:09 CEST
> DEBUG [2011-08-02 01:23:01,060]
> [org.apache.http.impl.conn.DefaultClientConnection:271] >> Host: <host>
> DEBUG [2011-08-02 01:23:01,061]
> [org.apache.http.impl.conn.DefaultClientConnection:271] >> Connection:
> Keep-Alive
> DEBUG [2011-08-02 01:23:01,061]
> [org.apache.http.impl.conn.DefaultClientConnection:271] >> User-Agent:
> Apache-HttpClient/4.1.1 (java 1.5)
> DEBUG [2011-08-02 01:23:01,061]
> [org.apache.http.impl.conn.DefaultClientConnection:271] >> Accept-Encoding:
> gzip,deflate
> DEBUG [2011-08-02 01:23:01,085] [org.apache.http.impl.conn.Wire:63] <<
> "HTTP/1.1 200 OK[\r][\n]"
> DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63] <<
> "Server: nginx/0.8.54[\r][\n]"
> DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63] <<
> "Date: Mon, 01 Aug 2011 23:23:01 GMT[\r][\n]"
> DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63] <<
> "Content-Type: text/plain[\r][\n]"
> DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63] <<
> "Last-Modified: Wed, 20 Jul 2011 14:39:57 GMT[\r][\n]"
> DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63] <<
> "Transfer-Encoding: chunked[\r][\n]"
> DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63] <<
> "Connection: keep-alive[\r][\n]"
> DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63] <<
> "Vary: Accept-Encoding[\r][\n]"
> DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63] <<
> "Expires: Wed, 31 Aug 2011 23:23:01 GMT[\r][\n]"
> DEBUG [2011-08-02 01:23:01,088] [org.apache.http.impl.conn.Wire:63] <<
> "Cache-Control: max-age=2592000[\r][\n]"
> DEBUG [2011-08-02 01:23:01,088] [org.apache.http.impl.conn.Wire:63] <<
> "Content-Encoding: gzip[\r][\n]"
> DEBUG [2011-08-02 01:23:01,088] [org.apache.http.impl.conn.Wire:63] <<
> "[\r][\n]"
> DEBUG [2011-08-02 01:23:01,088]
> [org.apache.http.impl.conn.DefaultClientConnection:249] Receiving response:
> HTTP/1.1 200 OK
> DEBUG [2011-08-02 01:23:01,089]
> [org.apache.http.impl.conn.DefaultClientConnection:252] << HTTP/1.1 200 OK
> DEBUG [2011-08-02 01:23:01,089]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Server:
> nginx/0.8.54
> DEBUG [2011-08-02 01:23:01,089]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Date: Mon, 01 Aug
> 2011 23:23:01 GMT
> DEBUG [2011-08-02 01:23:01,089]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Content-Type:
> text/plain
> DEBUG [2011-08-02 01:23:01,089]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Last-Modified:
> Wed, 20 Jul 2011 14:39:57 GMT
> DEBUG [2011-08-02 01:23:01,090]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Transfer-Encoding:
> chunked
> DEBUG [2011-08-02 01:23:01,090]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Connection:
> keep-alive
> DEBUG [2011-08-02 01:23:01,090]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Vary:
> Accept-Encoding
> DEBUG [2011-08-02 01:23:01,090]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Expires: Wed, 31
> Aug 2011 23:23:01 GMT
> DEBUG [2011-08-02 01:23:01,090]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Cache-Control:
> max-age=2592000
> DEBUG [2011-08-02 01:23:01,091]
> [org.apache.http.impl.conn.DefaultClientConnection:255] << Content-Encoding:
> gzip
> DEBUG [2011-08-02 01:23:01,091]
> [org.apache.http.impl.client.DefaultRequestDirector:477] Connection
> can be kept alive indefinitely
> DEBUG [2011-08-02 01:23:01,131] [org.apache.http.impl.conn.Wire:63] <<
> "600a[\r][\n]"
> DEBUG [2011-08-02 01:23:01,132] [org.apache.http.impl.conn.Wire:77] <<
> "[0x1f]"
> DEBUG [2011-08-02 01:23:03,838] [org.apache.http.impl.conn.Wire:63] <<
> "[\r][\n]"
> .... (Content)
> DEBUG [2011-08-02 01:23:03,839]
> [org.apache.http.impl.conn.SingleClientConnManager:267] Releasing connection
> org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter@2aa3873
> DEBUG [2011-08-02 01:23:03,839]
> [org.apache.http.impl.conn.SingleClientConnManager:285] Released connection
> open but not reusable.
> DEBUG [2011-08-02 01:23:03,839]
> [org.apache.http.impl.conn.DefaultClientConnection:152] Connection shut down
> ERROR [2011-08-02 01:23:03,840] [app] Exception downloading file
> java.io.EOFException
> at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:224)
> at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:214)
> at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:153)
> at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:75)
> at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:85)
> at
> org.apache.http.client.entity.GzipDecompressingEntity.getContent(GzipDecompressingEntity.java:63)
> at org.apache.http.util.EntityUtils.consume(EntityUtils.java:65)
> at
> org.apache.http.conn.BasicManagedEntity.ensureConsumed(BasicManagedEntity.java:98)
> at
> org.apache.http.conn.BasicManagedEntity.writeTo(BasicManagedEntity.java:115)
> at util.FileDownload.download(FileDownload.java:188) <--- my app
> Does this happen because the server doesn't specify the content length?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
