Hi, Spencer,
Your code looks okay to me. If you use a regular web-browser, with the
same proxy settings, can it always download the file? Also try with
"curl" or "wget" on the command line.
I have some code style nit picking below, but I don't think it will make
any difference. Good work on using InputStream.read( byte[] )! It
performs so much better than read() read() read() read()....!
-------------------------------------
I would do this before the "while( true )" rather than inside it:
byte[] buffer = new byte[12*1024];
I would try testing a few times without the "BufferedInputStream", but
just out of curiosity. BufferedInputStream tends to help a lot, but
since you're already using InputStream.read( byte[] ),
BufferedInputStream isn't going to be quite as amazing as he usually is.
Concerning code style, I think this looks cleaner (sorry to be so
obnoxious!):
long totalBytesRead = 0;
try
{
byte[] buf = new byte[ 4096 ];
int r = in.read( buf );
while ( r >= 0 )
{
if ( r > 0 )
{
totalBytesRead += r;
out.write( buf, 0, r );
}
r = in.read( buf );
}
}
finally
{
try { if ( in != null ) in.close(); } catch ( IOException ioe ) {}
try { if ( out != null ) out.close(); } catch ( IOException ioe ) {}
}
yours,
Julius
On Tue, 2006-01-08 at 10:05 -0700, Spencer Lee wrote:
> Hello,
>
> Just wondering if anyone has had problems retreiving data all the data of a
> large file.
>
> Here's basically the code I've been running...
>
> ...
> HttpClient httpclient = new HttpClient(params);
> httpclient.getHostConfiguration().setHost("myhost", 80, "http");
> httpclient.getParams().setCookiePolicy(
> CookiePolicy.BROWSER_COMPATIBILITY);
> httpclient.getHostConfiguration ().setProxy("webproxy", 80);
> ...
> // Create a method instance.
> GetMethod method = new GetMethod(myurl);
> DefaultHttpMethodRetryHandler retryhandler = new
> DefaultHttpMethodRetryHandler(10, false);
> HttpMethodParams method_params = method.getParams();
> method_params.setParameter(HttpMethodParams.RETRY_HANDLER,
> retryhandler);
> ...
> // executing the method
> int statusCode = httpclient.executeMethod(method);
>
> if(statusCode != HttpStatus.SC_OK) {
> System.out.println("ERROR: Status code " + statusCode);
> return;
> }
>
> // Read the response body.
> InputStream inpStream = method.getResponseBodyAsStream();
>
> BufferedInputStream bufinstrm = new
> BufferedInputStream(inpStream);
> long totalBytesRead = 0;
> while(true) {
> byte[] buffer = new byte[12*1024];
> int bytesRead;
>
> bytesRead = bufinstrm.read(buffer);
> if(bytesRead == -1) {
> System.err.println("Read -1. Read complete. Total
> Bytes Read: " + totalBytesRead + " bytes");
> break;
> }
>
> // bufotstrm.write(buffer, 0, bytesRead);
> totalBytesRead += bytesRead;
> System.err.println("Total Bytes Read: " + totalBytesRead + "
> bytes");
> }
>
> bufinstrm.close();
>
> // close
> method.releaseConnection();
>
>
>
> The byte size of the file is 36,197,082. But I am only able to retreival
> all 36,197,082 bytes once in awhile. On most occasions my
> BufferedInputStream (bufinstrm) will return a -1 to signal EOF before all
> bytes are read.
>
> eg.
> ...
> Total Bytes Read: 26796032 bytes
> Read -1. Read complete. Total Bytes Read: 26796032 bytes
> >
>
> Anyone come across anything similar to this before?
>
> Thanks!
>
> Spencer
>
> Also...
> * I'm using release commons-httpclient-3.0.1
> * I am not seeing any abnormal messages printed to the logs.
> * I am working through a web proxy server.
--
Julius Davies
Senior Application Developer, Technology Services
Credit Union Central of British Columbia
http://www.cucbc.com/
Tel: 604-730-6385
Cel: 604-868-7571
Fax: 604-737-5910
1441 Creekside Drive
Vancouver, BC
Canada
V6J 4S7
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
