Hi, This is Kishore who is working on async poc using mail id(kishore25ku...@gmail.com<mailto:kishore25ku...@gmail.com>). I work at adobe and we wanted to implement async support for jclouds library and contribute it back.
From the last discussion I was asked to get the performance numbers for the two approaches. Approach 1: Using Http Async Library Approach 2: Using Outputstream Test setup: 1. Both the s3 proxy server and test runner are running in same Docker container in azure west-us region. 2. Azure storage account is also residing in same west-us region. 3. A bucket is prepopulated with 100,000 files, each file of 1 MB size before test start. 4. The test runner sends unique requests to s3proxy to download files. Virtual Machine spec: CPU - 8 cores, Memory - 28 GB (Standard_D4 Azure machine) S3proxy is running with 1 jetty worker thread in all the scenarios. The payload size used is 1 MB file. Here are the performance numbers. Test Runner Threads Iteration Per thread Approach Avg response time (ms) 99%tile time (ms) Throughput (Requests / sec) 1 10,000 Async Http Lib 45 87 22 5 10,000 Async Http Lib 107 159 47 10 10,000 Async Http Lib 209 282 48 1 10,000 OutputStream 41 85 24 5 10,000 OutputStream 190 283 26 10 10,000 OutputStream 392 542 25 Summary: Under load Http Async Library approach is providing more throughput compared to Output stream approach. Both the approaches improve performance. The output stream approach can be used along with Http Async library approach which is giving around (3-5 ms) improvement in latency. Each approach is independent development. At this point I am keen to take up Http Async Library development. -- Thanks -- Kishore
