I notice you aren't checking your status code. Error response have a body/response entity too. Take a look at response.getStatusLine().getStatusCode() and getStatusReasonPhrase(). I bet the server is limiting you in the instances where you're seeing non-gziped content.
On Mon, Apr 1, 2013 at 10:42 PM, S.L <[email protected]> wrote: > Hi All, > > I use the following code and run the method multiple times, a few times I > get a response in gzip which is what I expect and a few time I get a > response that is completely different(non Gzip and html format) .However if > I download the same URL multiple times using Mozilla or IE I consistently > get the same GZIP response , Is this an error with the server I am trying > to reach to , or do I need to set parameters to get a consistent response ? > > Thanks. > > The URL I am trying to download is * > http://www.walmart.com/navigation6.xml.gz* , can you please let me know ? > Thanks > > > > > public static byte[] dowloadURL(URL urlToDownload) { > > InputStream iStream = null; > byte[] urlBytes = null; > > try { > > //HttpClient httpClient = new HttpClient(); > org.apache.http.client. > HttpClient httpClient = new DefaultHttpClient(); > > > HttpGet httpget = new HttpGet(urlToDownload.toString()); > > HttpResponse response = httpClient.execute(httpget); > > iStream = response.getEntity().getContent(); > > urlBytes = IOUtils.toByteArray(iStream); > String responseString = new String(urlBytes); > System.out.println(" >>> The response string for " > +urlToDownload.toString()+ " is " +responseString); > > } catch (IOException e) { > System.err.printf("Failed while reading bytes from %s: %s", > urlToDownload.toExternalForm(), e.getMessage()); > e.printStackTrace(); > // Perform any other exception handling that's appropriate. > } finally { > if (iStream != null) { > try { > iStream.close(); > } catch (IOException e) { > // TODO Auto-generated catch block > e.printStackTrace(); > } > } > } > > return urlBytes; > } >
