On Sat, 2015-05-23 at 22:09 +0200, Michael Osipov wrote:
> Hi,
>
> we are experiencing a (slight) performance problem with HttpClient 4.4.1
> while downloading big files from a remote server in the corporate intranet.
>
> A simple test client:
> HttpClientBuilder builder = HttpClientBuilder.create();
> try (CloseableHttpClient client = builder.build()) {
> HttpGet get = new HttpGet("...");
> long start = System.nanoTime();
> HttpResponse response = client.execute(get);
> HttpEntity entity = response.getEntity();
>
> File file = File.createTempFile("prefix", null);
> OutputStream os = new FileOutputStream(file);
> entity.writeTo(os);
> long stop = System.nanoTime();
> long contentLength = file.length();
>
> long diff = stop - start;
> System.out.printf("Duration: %d ms%n",
> TimeUnit.NANOSECONDS.toMillis(diff));
> System.out.printf("Size: %d%n", contentLength);
>
> float speed = contentLength / (float) diff * (1_000_000_000 / 1_000_000);
>
> System.out.printf("Speed: %.2f MB/s%n", speed);
> }
>
> After at least 10 repetions I see that the 182 MB file is download
> within 24 000 ms with about 8 MB/s max. I cannot top that.
>
> I have tried this over and over again with curl and see that curl is
> able to saturate the entire LAN connection (100 Mbit/s).
>
> My tests are done on Windows 7 64 bit, JDK 7u67 32 bit.
>
> Any idea what the bottleneck might me?
>
(1) Curl should be using zero copy file transfer which Java blocking i/o
does not support. HttpAsyncClient on the other hand supports zero copy
file transfer and generally tends to perform better when writing content
out directly to the disk.
(2) Use larger socket / intermediate buffers. Default buffer size used
by Entity implementations is most likely suboptimal.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org
